Is your slight increase in protection (minute increase really) greater than the contact risk from the person who has touched their contaminated mask/face then the handrail, shopping, whatever, that you touch immediately afterward?
Good practice dictates that after shopping, or touching any surface in a public area that you should wash/sanitise your hands.
If the whole premise for not encouraging the use of face masks is that a suspected proportion unknown of the population don't take hygiene seriously then I think masks are the least of our concern.
I suspect those who wear a face mask without taking care would also not take care without the mask and touch their phone, touch their face and contaminate surfaces in general.
At any rate, the outside of the mask is considerably less contaminated than the inside unless you are in an area with a high concentration of the virus such as a hospital. You should really not have any need to touch the inside of the mask even if you are fiddling and taking the mask off and on all the time.
I object to you quantifying the increase in protection as minute, we have no way to know, but if it's good enough for health professionals...
Another factor that is not discussed is the wearing any sort of face covering should lessen the viral load being shed from an infected person. Acknowledging the fact that disposable face masks won't prevent all infections, they may well have an impact on the severity of symptoms if a person infected by a face mask-wearer is subject to less of a load than if no face mask was being worn.