If that's the case then you must have stayed in a very small, very privileged pond.
As I said, I don't know a single Trust that has a policy against traveling in uniform (aside from aforementioned specialist areas) and I've worked in 20+ in just the last 11 years.
Here is an example of uniform...