Sunday's Signalling Failure at Machynlleth

I note from the BBC News that a "signalling fault" appears to have shut the whole Cambrian network on Sunday, with trains between Machynlleth and Aberystwyth, Pwllheli, and Shrewsbury all being cancelled.
https://www.bbc.co.uk/news/uk-wales-48817146

A signalling failure between train stations has caused disruption and delays to some rail services in Wales.

The fault happened between stations in Machynlleth and Pwllheli, in Gwynedd, and Aberystwyth.

Services between Shrewsbury and Aberystwyth were also cancelled today.

National Rail said the lines have now been reopened although trains may still be delayed or cancelled as services return to normal.

It added there would be a limited bus service running between Shrewsbury and Aberystwyth, and between Machynlleth and Pwllheli.

Click to expand...

Anybody know anything more? Something to do with the ETCS?

Today didn't start off too well either, with the early train from Pwllheli also being cancelled. I understand this was a train failure. Plenty of passengers on my train complaining about the 2-hour wait, and lots of late-running school kids (who didn't seem to be complaining so much!).

Interesting photo used on the BBC site - when did the Cambrian get OLE?

Yes there was a problem with the ERTMS system but I've heard no details of exactly what.
And yes this mornings problem was a train fault and there was no spare unit to cover it.

Belperpete said:

I note from the BBC News that a "signalling fault" appears to have shut the whole Cambrian network on Sunday, with trains between Machynlleth and Aberystwyth, Pwllheli, and Shrewsbury all being cancelled.
https://www.bbc.co.uk/news/uk-wales-48817146
Anybody know anything more? Something to do with the ETCS?

Click to expand...

Complete loss of the ERTMS system due to a telecoms hardware failure preventing the signalbox communicating with the outside world and vice versa meaning nothing could run.

Ahh the joys of modern technology a glitch and it all stops !

Considering the number of telephone equipped User Worked Crossings that Machy controls closing the line down seems justified.

Greg Read said:

Ahh the joys of modern technology a glitch and it all stops !

Click to expand...

This is a little unfair - a lot of technology can run in a degraded state. But railway technology has long been designed with failsafe principles in mind. Of course, things are more centralised than they used to be, but this is how it's meant to be.

What’s the degraded mode operation on an absolute block line if the bells/block instruments are out of action?

That seems like the analogous failure mode, would a 1920s signal box have fared any better?

RichardGore said:

What’s the degraded mode operation on an absolute block line if the bells/block instruments are out of action?

That seems like the analogous failure mode, would a 1920s signal box have fared any better?

Click to expand...

You resort to using the telephone......................
And when that doesn't work, you put the kettle on and put your feet up.

And this morning there were big delays when someone decided that the lights at Caersws Rd Crossing didn't apply to them and hit the barriers.

Drivers would've been authorised to pass signals at danger, proceed at caution (or up to a maximum of 50mph in an absolute block section). In some areas 'emergency block working' was a possibility (I think in recent times this method of work is/was a trial on the Western region only). But the job would still run albeit with delays.

There is a ridiculous 'eggs in one basket' culture with signalling schemes in general, not just with ERTMS, where a single point of failure can cripple a huge area (see York last summer, and other less newsworthy small scale stoppages).

In this case though pilotman working would have been required.

RichardGore said:

What’s the degraded mode operation on an absolute block line if the bells/block instruments are out of action?

That seems like the analogous failure mode, would a 1920s signal box have fared any better?

Click to expand...

30907 said:

In this case though pilotman working would have been required.

Click to expand...

Pilotman is only applicable to single lines. According to my old Kitchenside and Williams book block working would have been maintained by telephone, or if that had failed as well then it would be back to time interval working when you wait a certain period before sending the next train. Under total failure conditions, and I think also when using telephone block, trains would be restricted to travelling slowly enough to stop short of any obstruction. So the delay under such a failure would be considerable. Also if the boxes in question controlled several block sections, depending on the failure they might all become one long section with the signaller at the exit having to observe the train complete before the next one could enter.

So all in all this method of working would have a major impact on journey time and capacity, especially on today's railway which is generally much busier than 100 years ago. I'm not convinced that the total disruption from a few large and therefore newsworthy failures is greater than would result from many smaller but less reported failures under traditional methods. A modern signalling centre has several signallers on duty so less likely to suffer from staff shortfalls, and will probably have technicians on site to attend quickly to local problems. It should also be more easily protected from fire, intrusion and power/communications failures - though this may not be so true in practice!

edwin_m said:

Pilotman is only applicable to single lines.

Click to expand...

Which is why I mentioned it - there being only one double track section on the Cambrian.
But that doesn't negate your point about the delay that would be caused.

Llama said:

Drivers would've been authorised to pass signals at danger, proceed at caution (or up to a maximum of 50mph in an absolute block section). In some areas 'emergency block working' was a possibility (I think in recent times this method of work is/was a trial on the Western region only). But the job would still run albeit with delays.

There is a ridiculous 'eggs in one basket' culture with signalling schemes in general, not just with ERTMS, where a single point of failure can cripple a huge area (see York last summer, and other less newsworthy small scale stoppages).

Click to expand...

According to an earlier post, the signal box had lost the ability to communicate with the outside world. If that was the case then the signaller would not be able to authorise drivers to pass signals at danger. There appears to be a single point of failure that the design of the installation does not have any mitigation for.

There would always be a way - trains themselves are a means of communication and drivers could walk to the signal box to receive instructions. If a box was unable to function in that way it could be switched out and relatively normal running resumed with a longer block section.

Llama said:

There would always be a way - trains themselves are a means of communication and drivers could walk to the signal box to receive instructions. If a box was unable to function in that way it could be switched out and relatively normal running resumed with a longer block section.

Click to expand...

Increasingly, the control point for signals - signal box, ROC etc - is a long, long way from the signal. Wherry lines in East Anglia are soon to be controlled from Colchester for example. A long walk for a driver!
A lot of signal boxes have no switch out facility because they are never switched out!

Reliability of equipment and increased redundancy such that a single point of failure is eradicated or greatly mitigated has got to be the answer.

I was referring to historic practices on AB signalling rather than ROCs/PSBs etc.

Fringe boxes would never be switched out.

And yes I agree. Someone 'burnt some toast' a month or two after Manchester ROC was commissioned a few years ago. Not only all signalling staff but all TOC control staff were evacuated while trains stopped - unable to communicate with either signallers for operational instructions or TOC control staff to find out what was going on.

"Someone 'burnt some toast' a month or two after Manchester ROC was commissioned ..."

Yes, there's only so much you can do with design. I well remember a certain Signalling Centre being designed to be two hour fire resistant, LSZH cables, all cable routes fire stopped at walls, A/C system designed to keep smoke out etc. Even the toasters in the mess room were hard wired in to outlets rather than plugged in to sockets. Still didn't stop a member of staff who should have known better bringing a toaster to work and plugging it in on the operating floor with the inevitable result!

Another case where the CSR stand-by battery supplied inverter lasted for about 30 minutes instead of the designed 12 hours. Turned out that even though the sockets had special earth connectors turned through 90 degrees to stop appliances being plugged in "somebody" had bought suitable plugs and equipped stand alone fans - and even a kettle - with them "so that we had air and tea when the mains failed", obviously the load far outweighed the designed load and flattened the battery in no time at all.

You can't design out everything!

Llama said:

I was referring to historic practices on AB signalling rather than ROCs/PSBs etc.

Click to expand...

It would make no difference. If I cannot confirm with the next Box that I can give "Line Clear", because I have NO communications, then nothing is going into that Section, and I'm putting the kettle on.