I grew up near Stirling. There, if a bus arrived at a stop that already had a bus at it, people would be able to board both buses at once. In Edinburgh, buses wait until they're first in the queue before they lot people board. I think this is a significant factor in the slowness, to an extent that I'd say it outweighs the time saved by the exact fare system.
It can also negatively impact other traffic. Today I was driving into Edinburgh and was held up for almost 5 minutes behind buses at just one bus stop (a traffic island and some other road features meant a safe overtake wasn't possible). If the buses had boarded 2 or 3 at a time, there would've been a lot less time queuing.
Given that changing this would seem to be a win-win, I presume there's a strong reason for not doing it.
I think one good reason is a combination of busy bus stops and inconsistent/unpredictable dwell times. If you have three or four buses waiting to pull up to a stop (as frequently happens on Princes Street of an evening) and the buses behind start loading, once the first bus has gone it's difficult and potentially dangerous for them all to shuffle up part-way through the loading procedure. Then more buses come along and have to tag on to the end of the queue (which is further back than it would actually need to be if the buses
could all pull forward) which could in turn cause the inside lane to be occupied far further back than would be the case if the buses took it in turn to load - thus potentially blocking other bus stops, and/or junctions.
Also, on Princes Street at least, there is the tram line down the outside lane which reduces the opportunities for/advisability of buses ducking in and out of fragmented queues (it does happen, but not nearly as much as if the queues of buses weren't as orderly as they are now).
Blindtraveler's point about partially sighted users not having to wonder where the bus is actually going to stop to let them on is also a good one. Similarly, it means that people of limited mobility don't need to struggle up and down a line of buses to get on the one they want - it will eventually come to them.
Something else that could be done during the festival and other times that see a lot of tourists or infrequent bus users on the bus is having people at bus stops to guide passengers to actually get the correct bus and understand how payment works, rather than have them get on a bus and only then find out what they need to do, at the cost of the driver spending valuable time explaining this to them, or get the correct bus but attempt to get change of £10. This could be temporary staff or people who for the rest of year carry out other roles within Lothian Buses/TfE or the council.
That would certainly be helpful. They could also encourage the tourists to wait in an orderly fashion, in a way that doesn't block the pavement for other people - rather than hanging around in confused-looking clumps getting in everyone's way, as seems to be the usual practice! (I do wonder at people who fiddle around with Facebook and Twitter on the latest shiny smartphone while waiting at a stop, then when a bus arrives they have to engage the driver in a protracted conversation to find out something that Google or the LB app could have told them
while they were waiting. Sigh...)