A single petrol station has at least two pumps, most have four or six. When you queye behind another car in the petrol station, they will be gone in 5 mins or less. Waiting for an EV to charge, plus charging yours and you've wasted an hour doing nothing.
Google images and pretty much every publication and car dealer cites not being able to shift their EV stock. People don't want them. Lots of people who rent houses won't pay to install charging ports on their drive and same for people who rent or own properties in terraced streets.
Most (about 65%) of homes have off street parking, therefore there's the potential for a lot of cars to not need to use a public charging point for 95% (or more) of their miles.
Add to that, the fact that petrol cars are still very much in the majority of fuel type vehicles on the UK roads and it's not really a problem that the number of public chargers is only just around the same number as petrol stations in the UK.