• Our booking engine at tickets.railforums.co.uk (powered by TrainSplit) helps support the running of the forum with every ticket purchase! Find out more and ask any questions/give us feedback in this thread!

ECML/MML major power problems (09/08)

Status
Not open for further replies.

bramling

Veteran Member
Joined
5 Mar 2012
Messages
17,686
Location
Hertfordshire / Teesdale
There were at least 3 in the Hatfield station area including at Welwyn - Moorgate on the fast line. These were the reason for the massive delay on the 0952 Aberdeen - London service. I had heard about 23 on the great northern region but can’t validate this.

People were expecting the 717s to be the savour if the line but 313s are simple creatures. Easy to get going once the power is back. I wonder if a single type of train has ever caused to mainlines to collapse in this way before? Having a chuckle reading through the class 700 thread. “They will never fail one poster kept trying to convince us”.

Yes the grid cause the initial fail. Accept that. But the grid was totally restored by 1835. Shame the 700 are too complex for their own good.

“Nothing much bothers them”...

It just *had* to be the 700s (and 717s). Other types of train don’t seem to have been a problem, including other new ones like the 387s or 800s.

Is Thameslink just cursed?

One thing I will say, GTR have done well to pick things up such that today’s service doesn’t look to have been too bad. Considering how the pieces landed last night they have done well to run a pretty decent service from the start today. The same can be said for the day when the heat caused all the problems, where again they put in a creditable performance to get things more or less okay for the next day.
 
Last edited:
Sponsor Post - registered members do not see these adverts; click here to register, or click here to log in
R

RailUK Forums

Failed Unit

Established Member
Joined
26 Jan 2009
Messages
8,857
Location
Central Belt
“Nothing much bothers them”...

It just *had* to be the 700s (and 717s). Other types of train don’t seem to have been a problem, including other new ones like the 387s or 800s.

Is Thameslink just cursed?

One thing I will say, GTR have done well to pick things up such that today’s service doesn’t look to have been too bad. Considering how the pieces landed last night they have done well to run a pretty decent service from the start today. The same can be said for the day when the heat caused all the problems, where again they put in a creditable performance to get things more or less okay for the next day.

I suspect 313s rule on the Moorgate route this weekend.

It does concern me how the 700s will perform in the winter when ice causes chaos on the overhead. I hope they are not going to be sitting down for hours if we get cold mornings.
 

Bald Rick

Veteran Member
Joined
28 Sep 2010
Messages
29,070
Saw photos on the BBC showing people who detrained at Kentish Town and must have caused additional headaches.

Planned evacuation. But in any event nothing was running, so even if it had been ‘self evacuated’, it would have been less of a problem than normal.

Must have been odd in Control, to find that all the 700s had come to a stand at exactly the same time, over a wide area !
Certainly was! Whilst I wasn’t in control, I got the first message (from Mrs BR on a stranded train), and she said there was a complete power failure, but I could tell there wasn’t. And I could see that it was TL services with the problem. Didn’t know what to think!

As usual the 365s saved the day. I for one would have ended up spending the night in my office had it not been for them.

You wouldn’t. Thameslink were putting people in taxis, and even had on their disruption page that if you got a taxi they would reimburse it (I even get a receipt). Several of my friends did just that.
 

jon0844

Veteran Member
Joined
1 Feb 2009
Messages
28,013
Location
UK
I suspect 313s rule on the Moorgate route this weekend.

It does concern me how the 700s will perform in the winter when ice causes chaos on the overhead. I hope they are not going to be sitting down for hours if we get cold mornings.

I think they've coped well in previous cold weather, being old enough to have had those issues experienced, tested and sorted. The 717s must surely function the same?

(additional: good to see the evacuation was controlled and not people doing it themselves, given some of the Tweets posted yesterday)
 

ashkeba

Established Member
Joined
13 May 2019
Messages
2,171
Are the Desiro City units (700/717) uniquely vulnerable? Does the Great Eastern still have Desiros and did any of those sit down and it just get lost under the GTR hoopla?
 

Mitchell Hurd

On Moderation
Joined
28 Oct 2017
Messages
1,648
If I'm honest, I'm in full agreement with Failed Unit on this one. Like with the 09:52 Aberdeen to London train, is
pretty simple.
30 minutes to (incorrectly) perform boot up sequence.
another 6 1/2 hours trying to diagnose what went wrong

Sorry but there's the depot to diagnose what went wrong. Why did it have to be done with trapped trains behind?

Really, this train could have proceed at a very slow speed into Kings Cross once it was rebooted which then would have meant others like the 09:52 Aberdeen to London HST would not have so many hours late.
 

ashkeba

Established Member
Joined
13 May 2019
Messages
2,171
You wouldn’t. Thameslink were putting people in taxis, and even had on their disruption page that if you got a taxi they would reimburse it (I even get a receipt). Several of my friends did just that.
Not at KGX while I was there. GN staff were scarce (understandably) and no way to access the web with mobile networks in meltdown from the numbers. As I posted earlier, I'm expecting aggro over booking a hotel room but a taxi home would be about £700 so I've tried to save GTR some money here!
 

hwl

Established Member
Joined
5 Feb 2012
Messages
7,355
Are the Desiro City units (700/717) uniquely vulnerable? Does the Great Eastern still have Desiros and did any of those sit down and it just get lost under the GTR hoopla?
Desiro city have completely different traction equipment to desiro.
 

pompeyfan

Established Member
Joined
24 Jan 2012
Messages
4,181
On desiro classics it usually takes 15 minutes to do a unit reboot, however I’ve heard of a fitter reboot being done in seconds, something to do with cutting the VCB and battery at the same time. Is it possible to do this with Desiro cities?
 

mike57

Established Member
Joined
13 Mar 2015
Messages
1,633
Location
East coast of Yorkshire
An honourable score draw between fossil and renewables then. I wonder if it was more of a generation or connection problem in each case.
There were storms around Hornsea at the time of the problem, including one strike which I suspect to have been positive lightning based upon the fact that we saw the flash and heard the thunder, given the 5 sec rule distance would have been about right. This could cause the trip. The reason I suspect positive lightning is that it unusual to see a flash at this distance in daylight hours.

To quote from the wiki article on lightning

Positive lightning strikes tend to be much more intense than their negative counterparts. An average bolt of negative lightning carries an electric current of 30,000 amperes (30 kA), and transfers 15 coulombs of electric charge and 500 megajoules of energy. Large bolts of negative lightning can carry up to 120 kA and 350 coulombs.[40] The average positive ground flash has roughly double the peak current of a typical negative flash, and can produce peak currents up to 400,000 amperes (400 kA) and charges of several hundred coulombs.[41][42] Furthermore, positive ground flashes with high peak currents are commonly followed by long continuing currents, a correlation not seen in negative ground flashes.
 

Hadders

Veteran Member
Associate Staff
Senior Fares Advisor
Joined
27 Apr 2011
Messages
12,984
I was on an LNER train that pulled into the platform at WGC. A local door was opened and many passengers got off and made their own arrangements. I got off and got the bus back home. Had a conversation with the excellent train manager who was doing everything she could. She was about to start as passenger count as LNER were attempting to source buses/taxis for the remaining passengers.

Alongside us on the fast line was another LNER train which eventually made its way back to Stevenage. The passengers on that train were unable to get off at WGC as there are no fast line platforms.

It's shambolic that a power surge can cause an entire fleet of trains to fail simultaneously. Arguably GTR are responsible for the delay minutes as it was their trains that failed.
 

bramling

Veteran Member
Joined
5 Mar 2012
Messages
17,686
Location
Hertfordshire / Teesdale
Not at KGX while I was there. GN staff were scarce (understandably) and no way to access the web with mobile networks in meltdown from the numbers. As I posted earlier, I'm expecting aggro over booking a hotel room but a taxi home would be about £700 so I've tried to save GTR some money here!

Indeed. At the time I got to King’s Cross (just after 2100) the place was in total chaos. Pretty much every way into the station was shut, with no staff outside and the gates being guarded by either BTP or Network Rail security staff. PAs were playing staying that the station was closed and people were advised not to travel, yet through the glass people could be seen inside. I only managed to get in with some creativity, when I did it was quite obvious that people were essentially having to sort things out for themselves.

Fortunately I managed to find a train entirely off my own back - which is presumably why it ultimately left virtually empty, probably with many potential passengers still stuck outside the station.

My neighbour reports having to wait well over an hour for a taxi, with the bill coming to a cool £280.
 

Failed Unit

Established Member
Joined
26 Jan 2009
Messages
8,857
Location
Central Belt
Aside from the class 700...

I don’t think anyone will dispute that but do you think that taking over 6 hours to move trains is acceptable for a modern railway? Yes the route cause was national grid and many experts in the power industry could see it coming. But Siemens product let us down badly for service recovery? I am sure all the public facing communications will blame the initial fault - but I suspect Siemen’s staff will be looking into why their product messed up the network because of its inability to be reset.
 

Bald Rick

Veteran Member
Joined
28 Sep 2010
Messages
29,070
I think it’s fair to say Siemens / Cross London Trains will be invited for tea no biscuits.
 

ashkeba

Established Member
Joined
13 May 2019
Messages
2,171
As usual the 365s saved the day. I for one would have ended up spending the night in my office had it not been for them.
Were the 387s OK? I'm on one now and I think it's short formed (4 not 8) and no WiFi and it's whistling a bit like a door hasn't shut tight but that might just be the forecast high winds arriving early.
 

Failed Unit

Established Member
Joined
26 Jan 2009
Messages
8,857
Location
Central Belt
Were the 387s OK? I'm on one now and I think it's short formed (4 not 8) and no WiFi and it's whistling a bit like a door hasn't shut tight but that might just be the forecast high winds arriving early.

I suspect the more reliable stick is getting moved around. Likewise the other 4 coaches may not have made it to the starting point. I don’t know the weekend diagrams but lots of 387 / 365 on Peterborough services today. But that could be normal.
 

Sunset route

Member
Joined
27 Oct 2015
Messages
1,176
Sorry if this has been mentioned before but did the signalling continue to work if not why not.
K

On the Brighton Main line we had a momentary spike that caused random signals from Horley to Norwood Junction to revert to danger then back to proceed again, a few drivers with their noses pressed against their windscreens were the only casualties. Also the same spike caused the interlocking (computer based) in the Horsahm Area to fail but the system rebooted within two or three minutes and worked fine afterwards. I cant speak for other areas but NR has said public ally that all signalling that was effected restarted without any fuss. Also Lewes signalbox lost its domestic power supply but not the signalling power supply.
 

Howardh

Established Member
Joined
17 May 2011
Messages
8,120
I think the issue of no openable windows is going to gain salience. The problem is that once summer ends the issue gets forgotten until the following year.

There should be a health and safety pack issued for anyone unfortunate enough to have to use a 700. Cushions to to make the seats bearable, breathing apparatus for when the air conditioning fails (it’s gutless enough even when working correctly), and a supply of bottled water and a makeshift toilet like used in caravans for the newly found issue that you might be stuck on the train for multiple hours if the trains electronics locks out.
Seriously this is my worst nightmare when travelling by rail. I have prostrate issues - mine is enlarged which to put it politely means I have to go frequently. I use trains over coaches, for example, as I know they will have toilets. If I were stuck for 2-3 hours then I would be in serious pain. I would hope to be allowed access to a toilet to pee in a bottle or something and if not I don't know.
Guards on here will be able to explain the procedure for pax like myself who have health issues?
 

Dren Ahmeti

Member
Joined
17 Oct 2017
Messages
550
Location
Bristol
Let me answer a few questions:
The signalling and power to the OHL was restored very quickly, yet trains were unable to move due to the Siemens fleet of trains (700/717) exhibiting software issues after the power cut and a resultant drop in power frequency.

Drivers were advised to conduct a ‘Battery Off, Battery On, Pan Up, Pan Down, Reboot’ procedure, which didn’t work on nearly all of the approximately 17 trapped Siemens trains out of 23 trapped in total on the ECML.

Software updates (which took about 20 minutes to install) were then issued, and these attempts were made fruitless by train lighting failing on some units, which caused an immediate evacuation onto a standby unit (1Z99, possibly a 365).
 

LNW-GW Joint

Veteran Member
Joined
22 Feb 2011
Messages
19,555
Location
Mold, Clwyd
Andrew Adonis was on Newsnight last night and was blaming the National Grid system for the chaos, rather than the railway.
The only regulator in the frame threatening retribution is Ofgem rather than ORR.
But the railway should not have been as badly affected as it was - I'm sure there will be fallout with Siemens over the Desiro City performance.
As far as I can tell the WCML ran fairly well last night*, and that is the usual bolthole for ECML "do not travel" advice, at least for Leeds northwards.

*except it had West Midlands disruption all day with wires down between Coventry and Birmingham.
 

ashkeba

Established Member
Joined
13 May 2019
Messages
2,171
Andrew Adonis was on Newsnight last night and was blaming the National Grid system for the chaos, rather than the railway. […]
As far as I can tell the WCML ran fairly well last night*, and that is the usual bolthole for ECML "do not travel" advice, at least for Leeds northwards.
I'm sure I saw Euston was exit- only around the time I was making my decision whether to try alternatives or find a hotel room before it was more expensive than I could risk, but there was a lot of misinformation flowering last night, some of it apparently from the King's Cross tannoy as I mentioned earlier!

As for Andrew Adonis, I think he worked on some infrastructure report most recently, so I guess that's his strongest "I told you so" rather than pointing to his own time not grasping the beetle at the department of transport.
 

hwl

Established Member
Joined
5 Feb 2012
Messages
7,355
If I'm honest, I'm in full agreement with Failed Unit on this one. Like with the 09:52 Aberdeen to London train, is


Sorry but there's the depot to diagnose what went wrong. Why did it have to be done with trapped trains behind?

Really, this train could have proceed at a very slow speed into Kings Cross once it was rebooted which then would have meant others like the 09:52 Aberdeen to London HST would not have so many hours late.

If they train doesn't move you are going to be sorting out in situ.
 

Nicholas Lewis

Established Member
Joined
9 Aug 2019
Messages
5,996
Location
Surrey
Given the trains see, and are deigned to cope on a daily basis, voltage in the range of 15-27kv i don't see this is the cause. However, a massive swing in frequency is highly unusual and potentially the trains interference current monitoring system (to protect against falsely activating track circuits) has seen this as a traction converter package issue and shut the trains down and the software has prevented the driver from being able to reboot. Whatever, putting weather to one side, to have so many trains disabled simultaneously is unprecedented. Thus Siemens and GTR need to be held to account by one of the industry bodies or the politicians (Grant Shapps & George Freeman are nowhere in this) so the underlying cause is clearly understood and what is done about it. Whilst we don't want the grid to trip off it is a plausible scenario that the railway system has been designed for (and has historically dealt with adequately)but the nature of this incident meant frequency collapsed BEFORE the grid tripped and this is what is unusual.
 

UDC

Member
Joined
9 Aug 2019
Messages
8
I am left wondering just how much scenario testing is carried out on new trains before they come into service.
 

AlexNL

Established Member
Joined
19 Dec 2014
Messages
1,683
I am left wondering just how much scenario testing is carried out on new trains before they come into service.
An awful lot. But it you think you've tested everything, something unexpected will happen. :)

How did the Aventra based trains hold up? They're similar technology from the same era - did they also shut down en masse?
 

Aictos

Established Member
Joined
28 Apr 2009
Messages
10,403
London North Western wasn't accepting any tickets so if you live in St Albans for example, you were prevented by using the TOC to travel from London to St Albans via Watford Junction.

Not sure what kind of service they had but I'm sure they always accepted TL in the past?
 
Joined
27 Aug 2017
Messages
43
We really are in trouble if 950MWe of plant dropping can cause this.

Sizewell B could do that with a single unexpected trip, or a site trip at another nuclear station, for example.
950MW of wind disappeared, as well as around 500MW of gas and solar each according to gridwatch. Thats 2 GW of loss in a few minutes. That's fairly substantial. It also happened at roughly peak demand, meaning there was a lot less spare capacity in the system as backup.

just watching open trains and the area between Hatfield and Welwyn North is gridlocked and hasn't changed in the past 45 minutes since I opened the page. !!

It's hard to accept that a gust of wind caused a wind turbine to in turn cause this disruption... Surely they would have been designed to withstand winds well in excess of those we get here in the UK.
No, most wind turbines will feather at a certain speed to stop them spinning. This is usually a wind speed of 20-30 mph on more domestic units, 50-60mph on large scale installs.. Problem is, if they keep spinning too fast they tend to either catch fire in the generators, or start throwing blades, which is not an ideal situation.

Regarding electric cars as some have stated adding additional demand. It wouldn't be hard to program the latest smart chargers to start reducing output as the grid frequency changes. Maybe a gradual reduction to 49hz and any lower would completely stop charging. Most utility companies are also offering an E7 style or more flexible tariff that means most people would be avoiding charging at 1700 because the cost of electricity is just too high. Vehicle to Grid is still a way off, but if introduced would be an excellent way of providing a large amount of aggregate electric back to the grid to stabilise the situation before large scale rolling blackouts are even needed.
 
Last edited:

hwl

Established Member
Joined
5 Feb 2012
Messages
7,355
An awful lot. But it you think you've tested everything, something unexpected will happen. :)

How did the Aventra based trains hold up? They're similar technology from the same era - did they also shut down en masse?
Mostly on lines not affected by the power cuts but no reported problems
 

hwl

Established Member
Joined
5 Feb 2012
Messages
7,355
950MW of wind disappeared, as well as around 500MW of gas and solar each according to gridwatch. Thats 2 GW of loss in a few minutes. That's fairly substantial. It also happened at roughly peak demand, meaning there was a lot less spare capacity in the system as backup.


No, most wind turbines will feather at a certain speed to stop them spinning. This is usually a wind speed of 20-30 mph on more domestic units, 50-60mph on large scale installs.. Problem is, if they keep spinning too fast they tend to either catch fire in the generators, or start throwing blades, which is not an ideal situation.


Regarding electric cars as some have stated adding additional demand. It wouldn't be hard to program the latest smart chargers to start reducing output as the grid frequency changes. Maybe a gradual reduction to 49hz and any lower would completely stop charging. Most utility companies are also offering an E7 style or more flexible tariff that means most people would be avoiding charging at 1700 because the cost of electricity is just too high. Vehicle to Grid is still a way off, but if introduced would be an excellent way of providing a large amount of aggregate electric back to the grid to stabilise the situation before large scale rolling blackouts are even needed.
Gridwatch is a useful indicator but not always reliable and solar needs to be taken with pinch of salt as it is guestimated rather than measured like everything else, it is also snap shots every minute.

I'd believe the wind and gas numbers but not solar...
 
Status
Not open for further replies.

Top