Thameslink 'Congestion' Sat 23 Feb

[Bald Rick]

But if the brakes won't release, there's no point in coupling another 700 to it. In fact there's a risk that the brake fault could migrate to the rescue unit if the fault is in the dodgy unit's software.

If the preferred option IS to rescue by using another 700, our unit was right behind, so why couldn't we buffer up and push the errant unit out of the way?

Sounds like the "rescue by using another 700" option will always be scuppered by the operator's reluctance to use an in service train to rescue a failed one, or to turn a working unit into another failed one.

I’m not sure how many train rescues you’ve had to manage, but...

1) rescuing a failure using an in service train with passengers on board is rarely done, simply because then the rescuing train will also end up in the sidings with passengers on it.

2) faults transferring between units is known, and was rather common on electrostars and 319s. Less so on 700s (I’ve never heard of it, although that’s not to say it hasn’t happened).

3) train braking systems are designed such that if the brakes cannot be released, then another train can effectively by pass that problem. Almost all brake failures on multiple units are resolved this way. Very occasionally, the nature of the failure means that the rescued train has no brakes, hence the 5mph, and it must be rescued from the front.

4) taking 1 and 3 together, the very last thing you want to do is to use an in service passenger train to then push out a failure with no brakes at 5mph to a siding. With passengers on.

The contingency plan is something like:

a) driver fixes the fault with own knowledge (5 mins or so)
b) driver fixes the fault with help over the hotline from control (10 mins or so)
c) driver does hard reboot of unit (30 mins)
d) fitter fixes fault (coming from Farringdon, by tube or taxi if necessary, will be called out at step b)
e) train is rescued by closest available empty unit; usually one cancelled out of service. If failure is undrivable, rescue train must go on the front.

Others may correct me, but I can think of only 3-4 occasions where we have got to step e since the 700s came into service. This is rather better than for the trains they replaced.

 

[DynamicSpirit]

c) driver does hard reboot of unit (30 mins)

Re-booting takes 30 minutes???

 

[Surreytraveller]

Re-booting takes 30 minutes???

Yes..

 

[The Box Photter]

I’m not sure how many train rescues you’ve had to manage, but...

1) rescuing a failure using an in service train with passengers on board is rarely done, simply because then the rescuing train will also end up in the sidings with passengers on it.

2) faults transferring between units is known, and was rather common on electrostars and 319s. Less so on 700s (I’ve never heard of it, although that’s not to say it hasn’t happened).

3) train braking systems are designed such that if the brakes cannot be released, then another train can effectively by pass that problem. Almost all brake failures on multiple units are resolved this way. Very occasionally, the nature of the failure means that the rescued train has no brakes, hence the 5mph, and it must be rescued from the front.

4) taking 1 and 3 together, the very last thing you want to do is to use an in service passenger train to then push out a failure with no brakes at 5mph to a siding. With passengers on.

The contingency plan is something like:

a) driver fixes the fault with own knowledge (5 mins or so)
b) driver fixes the fault with help over the hotline from control (10 mins or so)
c) driver does hard reboot of unit (30 mins)
d) fitter fixes fault (coming from Farringdon, by tube or taxi if necessary, will be called out at step b)
e) train is rescued by closest available empty unit; usually one cancelled out of service. If failure is undrivable, rescue train must go on the front.

Others may correct me, but I can think of only 3-4 occasions where we have got to step e since the 700s came into service. This is rather better than for the trains they replaced.

Well, as a long-time victim of Thameslink and its predecessors, I've no experience of managing any train rescues, but I've lots of experience in being subject to inadequate management of such situations. I'd suggest that the contingency plan, if it gets beyond point (b) it's time to rustle up the nearest unit to effect a rescue. I didn't realise that it was a requirement to be a member of staff to post on this forum.

Allowing a unit to sit in the Core for 30 mins whilst it reboots is far too long, even at the weekend.

Likewise, the usefulness of Smithfield Sidings as a place to put a class 700 is long past. Only an eight car unit will fit in there and in emergencies the underused bay platforms at Blackfriars can be used. Smithfield is the obvious place to station a Thunderbird, being roughly mid-way along the network and slap bang in the middle of the Core.

 

[DynamicSpirit]

Re-booting takes 30 minutes???

Yes..

Out of interest, why? What happens in a re-boot?

I'm curious because the word 're-booting' seems to imply a software process similar to - say - re-booting Windows or turning on your phone. But it's a long time since I've encountered any hardware on which today's operating systems typically take more than 30 seconds or so to re-boot. Maybe a minute if you have something like SQL Server or some complex client-server software that needs to talk to a remote host to start up. But... 30 minutes???

 

[Tomnick]

Well, as a long-time victim of Thameslink and its predecessors, I've no experience of managing any train rescues, but I've lots of experience in being subject to inadequate management of such situations. I'd suggest that the contingency plan, if it gets beyond point (b) it's time to rustle up the nearest unit to effect a rescue. I didn't realise that it was a requirement to be a member of staff to post on this forum.

Allowing a unit to sit in the Core for 30 mins whilst it reboots is far too long, even at the weekend.

Likewise, the usefulness of Smithfield Sidings as a place to put a class 700 is long past. Only an eight car unit will fit in there and in emergencies the underused bay platforms at Blackfriars can be used. Smithfield is the obvious place to station a Thunderbird, being roughly mid-way along the network and slap bang in the middle of the Core.

I don’t understand, in this case, what a Thunderbird loco could do that the unit behind couldn’t?

 

[sga962]

DOO fault is what the internal emails say in this sort of incident so while it may not be specific enough for some people it is often as much information as most have.

In this particular incident it started out as a DOO fault and while it is possible to run such a service calling only at stations with sufficient staff for despatch, it is of course preferable to rectify the fault if possible and to run as close to booked as possible.
Fitters were onsite fairly quickly
Once the service was definitely cancelled passengers were detrained onto platform A

The brake release issues was actually resolved not long before the rescue unit arrived and once coupled they went to Elephant and castle for a while.

 

[bramling]

Out of interest, why? What happens in a re-boot?

I'm curious because the word 're-booting' seems to imply a software process similar to - say - re-booting Windows or turning on your phone. But it's a long time since I've encountered any hardware on which today's operating systems typically take more than 30 seconds or so to re-boot. Maybe a minute if you have something like SQL Server or some complex client-server software that needs to talk to a remote host to start up. But... 30 minutes???

It’s effectively a complete trip and reset of everything on the train. However 30 minutes does sound a lot, considering 1990s trains can do it in a couple of minutes.

 

[carriageline]

But... 1990s trains had a fraction of the technology that the 700s have. The 700s deal with 2 completely different signalling systems, as an example!

 

[The Box Photter]

I don’t understand, in this case, what a Thunderbird loco could do that the unit behind couldn’t?

Apparently, the operators won't use a unit that's full of self-loading cargo to push a failed unit out of the way (even though the units are designed to be able to do that). It's quite rare to have a unit in the Core that isn't in service and full of passengers, so the units either side of the failed one would have been no use in effecting a rescue. Instead we sat in the tunnel for about 40 minutes until we could run past "wrong line".

Had a Thunderbird been available at Smithfield Sidings, they could have allowed all the in service trains ahead of the failed one to vacate the Core, then drop the loco onto the failed unit. That should be achievable within 15 - 20 minutes and should have happened once the driver of the failed unit had been unable to resolve the fault themselves. In fact if the Thunderbird loco was already sitting "warmed up", it could have been positioned to move onto the failed unit after about 10 minutes.

There seems to be a willful determination to make things more complicated than they need be. And don't get me started on the ludicrous 30 minute reboot time. Who thinks that's an acceptable "feature" of a modern train that's supposed to be delivering a 90 second headway service?

 

[The Box Photter]

But... 1990s trains had a fraction of the technology that the 700s have. The 700s deal with 2 completely different signalling systems, as an example!

As Scotty famously said, "The more you complicate the plumbing, the easier it is to stop up the drain".

The second signalling system, that'll be the one they're not using yet will it? That'll be the one they need to deliver the tube-train frequency service that no-one asked for and isn't likely to be deliverable in the foreseeable future is it?

 

[carriageline]

You make completely valid points Mr Photter. But...

Dedicated Thunderbird locos are a rarity nowadays, across the network. Plus, most locos (if not all) are not prohibited through the core, especially with the AC on.

No one wants to pay for them either, for the (let’s face it) rare time it would be needed. I know there was talk of getting thunderbirds on the MML, but the type of loco being talked about wasn’t able to be used in the core, at a minimum it required the AC isolated if I remember correctly. So it’s primary use would be for use north of the core. Whether that happened or not, I am unsure.

The “30 minute” reboot is a complete hard reboot, starting from scratch. I believe there is a soft reboot, which is much shorter. I haven’t seen the hard reboot used much in traffic.

 

[Bald Rick]

I didn't realise that it was a requirement to be a member of staff to post on this forum

No it isn’t, and quite rightly so. Neither is it a requirement to be a member of staff to ask questions about why things happen. However it is generally considered to be polite not to consistently rubbish the work done by people who are members of staff who might just be doing their very best in difficult circumstances.

Apparently, the operators won't use a unit that's full of self-loading cargo to push a failed unit out of the way (even though the units are designed to be able to do that). It's quite rare to have a unit in the Core that isn't in service and full of passengers, so the units either side of the failed one would have been no use in effecting a rescue. Instead we sat in the tunnel for about 40 minutes until we could run past "wrong line".

Had a Thunderbird been available at Smithfield Sidings, they could have allowed all the in service trains ahead of the failed one to vacate the Core, then drop the loco onto the failed unit. That should be achievable within 15 - 20 minutes and should have happened once the driver of the failed unit had been unable to resolve the fault themselves. In fact if the Thunderbird loco was already sitting "warmed up", it could have been positioned to move onto the failed unit after about 10 minutes.

How long do you really think it would take for a ‘warmed up’ thunderbird to be on the front of a train from Smithfield? Because it isn’t 10 minutes. Given the nature of any failure (as per the steps I listed earlier) it would be at least 10 minutes before it would be decided to activate that as a plan, another 10 minutes to clear the core Southbound, probably half an hour to get the trains sat at City TL and Blackfriars northbound platforms out of the way, and the the thunderbird could have got out. By then, 99 times out of 100 the train failure would have long since been on its way.

Do you really, really, think that these things haven’t been thought through, modelled, identified the best and most practical way to solve, written into procedures and practised? Particularly given the incident at St Pancras a few summers ago?

The “30 minute” reboot is a complete hard reboot, starting from scratch. I believe there is a soft reboot, which is much shorter. I haven’t seen the hard reboot used much in traffic.

I reckon a hard reboot happens somewhere in traffic about once every couple of weeks. In the core I’ve known it happen a handful of times, most of which were in the first few weeks of 700 operation.

 

[ComUtoR]

The “30 minute” reboot is a complete hard reboot, starting from scratch. I believe there is a soft reboot, which is much shorter. I haven’t seen the hard reboot used much in traffic.

I reckon a hard reboot happens somewhere in traffic about once every couple of weeks. In the core I’ve known it happen a handful of times, most of which were in the first few weeks of 700 operation.

I have rebooted a 700 three times. [caveat]You are both quite knowledgable and have 'inside knowledge' and because I am forked off with being called out for questioning people when I have an enquiry[/caveat] You are also two people who generally understand and have been very very helpful in the past BUT...

WTF is a 'hard reboot' compared to a 'soft reboot' ? Are you talking about a simple key off/on (cab reset) as a 'soft reboot' ? I have heard of a reboot taking an extended time but I have the reboot procedure in front of me and I can only see the one reboot procedure. I know it happens as I have anecdotal evidence of it but I only know one way of doing it (kinda)

cheers in advance.

 

[The Box Photter]

I have rebooted a 700 three times. [caveat]You are both quite knowledgable and have 'inside knowledge' and because I am forked off with being called out for questioning people when I have an enquiry[/caveat] You are also two people who generally understand and have been very very helpful in the past BUT...

WTF is a 'hard reboot' compared to a 'soft reboot' ? Are you talking about a simple key off/on (cab reset) as a 'soft reboot' ? I have heard of a reboot taking an extended time but I have the reboot procedure in front of me and I can only see the one reboot procedure. I know it happens as I have anecdotal evidence of it but I only know one way of doing it (kinda)

cheers in advance.

I think a "hard reboot" involves taking the batteries out and blowing on them before putting them back in .... I could be wrong though.

 

[The Box Photter]

You make completely valid points Mr Photter. But...

Dedicated Thunderbird locos are a rarity nowadays, across the network. Plus, most locos (if not all) are not prohibited through the core, especially with the AC on.

No one wants to pay for them either, for the (let’s face it) rare time it would be needed. I know there was talk of getting thunderbirds on the MML, but the type of loco being talked about wasn’t able to be used in the core, at a minimum it required the AC isolated if I remember correctly. So it’s primary use would be for use north of the core. Whether that happened or not, I am unsure.

The “30 minute” reboot is a complete hard reboot, starting from scratch. I believe there is a soft reboot, which is much shorter. I haven’t seen the hard reboot used much in traffic.

Aren't class 73s cleared to run through the Core? I've seen them on engineering trains at Farringdon and they can use DC/diesel power. There shouldn't be any clearance issues with the OHLE otherwise the normal 700s would be arcing all over the place. I know such locos are a rarity now and need paying for, but the requirement to provide one can be drawn into the franchise/management contract. Any operator who doesn't want to meet the cost needn't bother bidding.

 

[Randomer]

I know there will be a robust safety case in place for Thameslink but I can't help wonder how different it is that SBB maintain rescue and firefighting locomotives with a defined response time for any tunnel in the country and why it is considered unnecessary here.

Obviously there is a considerable cost to it, SBB having had the last of a new batch delivered last year, but there risk assessment for tunnels must make it a requirement that we don't have. Type of traffic allowed through along with the length of the tunnels involved possibly?

 

[JamesT]

Out of interest, why? What happens in a re-boot?

I'm curious because the word 're-booting' seems to imply a software process similar to - say - re-booting Windows or turning on your phone. But it's a long time since I've encountered any hardware on which today's operating systems typically take more than 30 seconds or so to re-boot. Maybe a minute if you have something like SQL Server or some complex client-server software that needs to talk to a remote host to start up. But... 30 minutes???

If you want to see slow boot times, you’ll want to try enterprise network gear. Rebooting a high end switch can take 5-10 minutes, big chassis models can take even longer as each line card needs rebooting in turn. They tend towards correctness rather than speed.
Talking of more normal servers, the POST process where the hardware is tested seems to take several minutes, before the OS even has a chance to start booting.

 

[atraindriver]

<deleted>

 

[Bald Rick]

I have rebooted a 700 three times. [caveat]You are both quite knowledgable and have 'inside knowledge' and because I am forked off with being called out for questioning people when I have an enquiry[/caveat] You are also two people who generally understand and have been very very helpful in the past BUT...

WTF is a 'hard reboot' compared to a 'soft reboot' ? Are you talking about a simple key off/on (cab reset) as a 'soft reboot' ? I have heard of a reboot taking an extended time but I have the reboot procedure in front of me and I can only see the one reboot procedure. I know it happens as I have anecdotal evidence of it but I only know one way of doing it (kinda)

cheers in advance.

Beyond my knowledge I’m afraid, but will ask a man who knows when I see him later his week after a few shandies. But I *think* the soft reboot is key off and reset aux.

 

[ComUtoR]

Beyond my knowledge I’m afraid, but will ask a man who knows when I see him later his week after a few shandies. But I *think* the soft reboot is key off and reset aux.

Cheers, I'd be most grateful.

Key On/Off does clear some faults; just like any other computer..
There is a soft/hard off for power (AC/DC system)
And then there is a full reboot (probably the '30' minutes one.)

 

[Supercoss]

Using a ‘ Thunderbird’ or rescue loco adds further issues as only class 73 are permitted between Kentish Town and Farringdon but with overhead line switched off due to height clearance issues. An adaptor coupler ( buckeye to Dellner) also required ( kept at West Hampstead or Farringdon). The ability to run round the failure ( in this case at Low Level) by running bi Directional Midland Road scissor crossover to Clerkenwell crossover would be lost if overhead line switched off on Up Moorgate 73/9 are mandated as rescue locos despite GTR having a 73/1 on its books (202) due to extra h p required for gradients. Pushing out failure to Herne Hill Siding or hauling out to Kentish Town at *Five* mph very disruptive.

 

[The Box Photter]

OK. So the consensus seems to be that whilst a failed class 700 can be pushed/pulled out of the way by another class 700, this can only be achieved if the rescue unit is ECS. The likelihood of there being an available class 700 in the vicinity of the failed unit (especially if the failure happens in the Core and at peak times) is slim. This means that all the pressure is put on an increasingly stressed driver to attempt to sort out the problem themselves and potentially with telephone assistance (assuming they can speak to a fitter).

Once the driver has exhausted all options, they have to wait for a fitter to get there from Farringdon (assuming they can get there by train on the other line or by taxi, which might be in the rush hour).

If the fitter can't resolve the problem once on site, someone has to make a decision to source a rescue unit (presumably from Cricklewood/Three Bridges/Bedford/Hornsey) or to create a rescue unit by de-training passengers from an in service train somewhere. The unit has to travel to the site, couple up and hope it can tow/push the failed one to a safe place. I can see why all this can easily extend beyond 30 minutes from the first failure time.

In the meantime, the operation through the Core is severely compromised, potentially with passengers stuck on trains, in tunnels either side of the failure with no means of communicating with their workplace/family/doctor's surgery/hospital etc to explain why they're delayed.

The simple answer to me still seems to be to have a Dellner-fitted rescue loco on 24 hr standby, paid for by the TOC as part of the franchise contract. As I've said, any potential TOCs who don' want to pay the cost of a rescue loco need not apply. If 73s can only run through the Core with 25kV isolated, then lay 3rd rail through the Core from Farringdon to Dock Junction so they can operate off the juice.

There needs to be some out of the box thinking to counteract the fatal flaw that the Core is 2 track only and that St Pancras LL is only two platforms.

 

[Surreytraveller]

OK. So the consensus seems to be that whilst a failed class 700 can be pushed/pulled out of the way by another class 700, this can only be achieved if the rescue unit is ECS. The likelihood of there being an available class 700 in the vicinity of the failed unit (especially if the failure happens in the Core and at peak times) is slim. This means that all the pressure is put on an increasingly stressed driver to attempt to sort out the problem themselves and potentially with telephone assistance (assuming they can speak to a fitter).

Once the driver has exhausted all options, they have to wait for a fitter to get there from Farringdon (assuming they can get there by train on the other line or by taxi, which might be in the rush hour).

If the fitter can't resolve the problem once on site, someone has to make a decision to source a rescue unit (presumably from Cricklewood/Three Bridges/Bedford/Hornsey) or to create a rescue unit by de-training passengers from an in service train somewhere. The unit has to travel to the site, couple up and hope it can tow/push the failed one to a safe place. I can see why all this can easily extend beyond 30 minutes from the first failure time.

In the meantime, the operation through the Core is severely compromised, potentially with passengers stuck on trains, in tunnels either side of the failure with no means of communicating with their workplace/family/doctor's surgery/hospital etc to explain why they're delayed.

The simple answer to me still seems to be to have a Dellner-fitted rescue loco on 24 hr standby, paid for by the TOC as part of the franchise contract. As I've said, any potential TOCs who don' want to pay the cost of a rescue loco need not apply. If 73s can only run through the Core with 25kV isolated, then lay 3rd rail through the Core from Farringdon to Dock Junction so they can operate off the juice.

There needs to be some out of the box thinking to counteract the fatal flaw that the Core is 2 track only and that St Pancras LL is only two platforms.

You create a rescue unit by terminating and detraining the nearest compatible unit, which would usually be the one in front or behind. As soon as the signaller is aware of a train in difficulty, following trains are held in platforms to avoid passengers being stuck between stations.
There's no need to have a rescue loco in standby - as soon as a train is cancelled, everyone would be saying 'why is there a driver sitting doing nothing on the rescue unit, when they should be driving trains?'

 

[03_179]

Serious question, could a couple of Class 73 locos not be converted to couple to 700 (and 717) stock and perform Thunderbird duties?

One at say the sidings between City T/L and Farringdon and maybe one at Cricklewood or Three Bridges ?

 

[carriageline]

Serious question, could a couple of Class 73 locos not be converted to couple to 700 (and 717) stock and perform Thunderbird duties?

One at say the sidings between City T/L and Farringdon and maybe one at Cricklewood or Three Bridges ?

But once again, you have to isolate the AC through the core when a class 73 goes through. That will make the situation 10x worse as nothing can move

Extending the DC is of course an option, but that brings its own issues. Blackfriars has its own issues where the DC/AC can interact with each other, even though the AC stops at city!

 

[The Box Photter]

You create a rescue unit by terminating and detraining the nearest compatible unit, which would usually be the one in front or behind. As soon as the signaller is aware of a train in difficulty, following trains are held in platforms to avoid passengers being stuck between stations.
There's no need to have a rescue loco in standby - as soon as a train is cancelled, everyone would be saying 'why is there a driver sitting doing nothing on the rescue unit, when they should be driving trains?'

The major problem with that is the probability that the nearest unit is already past the previous station where passengers could be de-trained, as was the case on Saturday. I was on that train. We couldn't move forwards to St Pancras because the platform was occupied by the failed unit and we couldn't move backwards to Kentish Town because there was another train behind us in the section between Kentish Town and Dock Junction.

I don't know the location of the train immediately in front of the failed unit, but I can imagine how happy passengers on it would have been if they'd been told "Please get off at the next station so we can use this unit to go back and rescue a broken down one, oh and by the way, we don't know when there'll be another train coming through to pick you up to continue your journey".

It just doesn't make sense.

 

[Surreytraveller]

The major problem with that is the probability that the nearest unit is already past the previous station where passengers could be de-trained, as was the case on Saturday. I was on that train. We couldn't move forwards to St Pancras because the platform was occupied by the failed unit and we couldn't move backwards to Kentish Town because there was another train behind us in the section between Kentish Town and Dock Junction.

I don't know the location of the train immediately in front of the failed unit, but I can imagine how happy passengers on it would have been if they'd been told "Please get off at the next station so we can use this unit to go back and rescue a broken down one, oh and by the way, we don't know when there'll be another train coming through to pick you up to continue your journey".

It just doesn't make sense.

Safety is the most important thing. Send a rescue train to retrieve a stranded one, the biggest risk is it ploughs into the rear of the stranded one. You reduce the risk of killing people by removing people from the equation. Yes, it inconveniences people, but that is preferable to killing them. The reason why it is done this way is because at some point in the past, people will have been killed due to this scenario, and the rules would have been changed to prevent a reoccurrence

 

[The Box Photter]

Safety is the most important thing. Send a rescue train to retrieve a stranded one, the biggest risk is it ploughs into the rear of the stranded one. You reduce the risk of killing people by removing people from the equation. Yes, it inconveniences people, but that is preferable to killing them. The reason why it is done this way is because at some point in the past, people will have been killed due to this scenario, and the rules would have been changed to prevent a reoccurrence

That point in the past will have been at a time when trains were much less structurally safe that they are now. When they were mostly all hauled by a heavy steam loco which is difficult to control with great precision. I can fully understand that it would be dangerous to send a loco and train charging off down the track without knowledge as to exactly where the stranded train is and in the knowledge that the rear coach of the stranded train was constructed from timber of indeterminate quality, mounted on a steel chassis, coupled to the next carriage by a chain link coupler.

But that's not the scenario we're talking about here. The location of the failed unit was known to the millimeter and the units in question are constructed to the highest possible current standards for withstanding collisions. The track is signaled to the latest Network Rail standards and I'm led to believe that the drivers can stop them exactly where needed, because their route knowledge and train handling skill is valued at around £70,000 per annum, so any risk to passengers would have been almost non-existent.

By your argument, we should be seeing trains ploughing into each other all the time with passengers being thrown across the tracks on a regular basis. The 21st century railway is being operated as though it's still the 19th century.

 

[westcoaster]

Serious question, could a couple of Class 73 locos not be converted to couple to 700 (and 717) stock and perform Thunderbird duties?

One at say the sidings between City T/L and Farringdon and maybe one at Cricklewood or Three Bridges ?

Tests with a 700 and 73 show a 73 does not have the required brake force by its self to handle the 700 on the steep gradients in the core, even a 67 can not handle the 700 Hence why it has to be another unit be it a 319/377/387 or 700.

With regards to the doo reset if a soft reset flicking out the mcb's does not solve the issue, the unit will need a battery off battery on reset done. It this is done the unit will need a full auto brake test and evc tests done before moving off. This is what takes time.