The short story being that in the eighties BR was forced to replace its aging three car DMMUs and longer Mk1 rakes with two car 15x. In order to squeeze passengers into the remaining coaches something had to give and the guard's van was lost. So you end up with the 150, a unit with absolutely no space at all for large cases (though to be fair to them, at least they have four bike spaces).
You don't even need to go back to the 1980's, since 2000 rail use until now (comparing the last quarter there's data for and the comparable quarter from 2000) has grown by about 2/3'rds (for every 100 passengers in 2000 there's 162 in 2023). If we were to look at pre COVID numbers it's even higher (185 passengers in 2019) and higher than that it where we want to be heading.
That puts a large amount of pressure on the TOC to provide extra seats (given the limited ability to build significant extra infrastructure capacity within 23 years), that leads to conflict where that hasn't been provided (often at the behest of the government).
Ultimately we need more rail capacity and that means a lot more infrastructure spending across the board, so HS2, Crossrail 2, electrification, platform lengthening, NPR, etc. to name just the well-known ones which should be much further on than they are.
Paired with that there should be longer trains, for example XC should be running a mixture of 5 and 9 coach units, not a mixture of 4 and 5 coaches, Northern should (except for some of the branch lines) be working towards 8 or 12 coach trains starting to be the norm at peak hours not 6 or 8 (or less).
Due to the fact we'll want to be heading to over double the capacity we had in 2000, there should be a 30 year plan with what would be needed to allow rail use to double again over that timeframe.
For some areas that'll be longer trains, for some lines that may actually not be anything, for other areas that may require doubling of tracks, new lines, reopenings, new junctions/stations to create easier journey routes (especially where it removes the need to go into a city and back out, which eats up valuable capacity), new tram/light rail networks, and so on.
Without it the same arguments about lack of space for those who wish to use rail will come around again and again and again. Each time they come around the anger will get worse as there's even less spare capacity. Yet both sides of the debate will be putting their energies into fighting each other rather than making for what would be needed to satisfy everyone - that being significantly more capacity.
Yes the Elizabeth Line cost a lot, went over budget, was late, etc. (I don't think we know how to not do that with road or rail projects, so none of that should be an issue), however it's significantly improved rail for quite a large area.
Repeat that with other projects which create a significant step change in capacity and we could see things improve across large areas of the country.
For example remove the long distance trains from Manchester's existing platforms and you allow more local services to be run with longer trains. As the local trains need less turn around time the removal of one long distance train could allow quite a lot more services to see improvements.
Remove two ICWC services (an hour of platform use) and it could allow 4 existing services (currently 6 coaches long) to be lengthened to 12 coaches. However those services were already using up platform capacity, if they were sharing a platform with another service that could allow those services to be lengthened from 4 coaches to 8 coaches.
In doing so you've just doubled the capacity of 8 services, whilst at the same time reduced the congestion in the approaches as there's 4 less train movements. It also allows for some ICWC services to be retained to cater for residual movements (although better still would be to allow those retained services, as well as other intercity services like XC, to also use new platforms to further improve the capacity uplift)
Likewise the construction of Crossrail 2 would allow a lot of the SWML (beyond Woking) to see an uplift in capacity from the extra 8 Waterloo services which would be delivered from the removal of some of the metro services from Waterloo as they would then be in the Crossrail 2 tunnels. On paper that's about (off peak) 3tph for every 2tph a station has (in reality is likely that capacity issues will mean that isn't as clear cut as that - for instance it's likely that Portsmouth wouldn't see any uplift without significant improvements with Havant being the best end of the line which could be achieved), that's a significant uplift in capacity across quite a large area.
A doubling of rail capacity would likely mean 20% of miles traveled being undertaken by train, which would still leave ~70% being done by road (so still a lot of people could only ever drive, so no forcing of people out of their cars). However with that sort of modal shift road congestion would be mostly limited to very busy periods (actually the peak hour, rather than between 6:30 and 9:30) and so benefit everyone.
It would likely also, even allowing for all cars to be EV's, require less energy to move those people by train than by car. This would mean that there would be more energy to do other things (such as air source heat pumps) without having to build quite so much energy infrastructure.
Of course for many currently alive 30 years could well be beyond their lifespan, so why bother. It's certainly will beyond the planning government thinks about and so they won't care either.
The reason that it's needed, is whilst things may well change (for instance teleportation may become a thing) having a plan for what's the best thing to do over the next 30 years doesn't fix you into actually delivering it if the needs change over that timeframe. It would also need to be a rolling plan, so each parliament would have to extend it by another (say) 5 years.
Whilst the delivery of stuff in 30 years time certainly isn't a vote winner, delivering what was promised (maybe even adding in a small extra scheme, or delivering something early) would attract more votes than delays and cuts to what was promised.
It would certainly need some blue sky thinking, for instance do you build the line between Okehampton and Plymouth to allow passenger numbers to build up before then building the Dawlish Avoiding Line to them further boost journey opportunities between Exeter and Plymouth and where does that sit in the plan for building a Wales and West HS line and with a new rail bridge across the Tamar (and do you even bypass some services around Plymouth so that bridge could be away from Plymouth)?
As there's no way that you could just lengthen the 80x's to double the capacity between London and Wales/Southwest.
Likewise do you reopen what is currently the Watercress Line so you can bypass Basingstoke and the congestion there - is that ultimately the future for most heritage lines, that they become part of the mainline network again?
Does anyone disagree with this, and should we just fight it out for ever more valuable train space as they get more and more overcrowded?
Now I do agree that having longer trains will solve a lot of problems. What won't wash with the bean counters is ordering longer units for bikes. Not unless the bikes are paying full fares.
Whilst bikes are unlikely to be paying full fares, the extra capacity created by longer trains would attract more passengers without actually costing the same proportion more.
For example, a 4 coach train may provide (say) 90% more seats as well as extra luggage and cycle space than a 2 coach train, however whilst some costs would increase by 100% (lease costs, maintenance, track access charges, etc.) the staff costs don't (you still need one driver and one guard whether it's 1 coach long or 12 coaches long).
As such, if a given train is over (say) 90% full 90% of the time then there's a very good chance that lengthening the train would result in more people using the service and so would likely cover the extra costs.
In fact there's a fairly good chance that it could result in it reducing the subsidy required to run the railways (obviously not by much but by a little). In that there's a lot of fixed costs, for example if a line has 15 trains a day with 2 coaches or 15 trains a day with 4 coaches it's unlikely to alter the number of inspections that track has, it's unlikely to impact the number of times that deicing trains run during the winter, it's unlikely to change how often stations get painted or platforms resurfaces.
Even if the trains were to be 8 coaches rather than 3 it's unlikely that many of those sorts of costs would increase and if they do not buy very much.
We are at the point where, if we want to reduce the cost to the government, we need to look at how we run more coaches over sections of track which being in the most extra income without having a material cost in providing the infrastructure. Ultimately that's looking at lines which have short trains or infrequent trains and asking what needs to happen to make more people use these rail services and can that happen in such a way that the extra income more than covers the extra costs.