Signal box failure - theoretical question

[tygar2]

When was the last time a load of trains got stuck for 6 hours?
Its hardly 'real world' is it!

After six hours I would be leaving the train!

 

[Tomnick]

No it doesn't. Intermediate block signals, and signals controlled from a signal box that has been confirmed to be closed may be passed under the driver's own authority. In practice, I am guessing, with almost universal train radio coverage and mobile phones everywhere, it's unlikely you'd ever be completely stranded without communication, even if you have to walk a bit to make contact.

If you can't communicate with the controlling signalman though (because the box is closed!), it doesn't matter who else you can communicate with. No-one else, in this scenario, can authorise you to pass the signal at danger.

I've nearly been there as a signalman (if a fault had occurred a few minutes earlier, I'd have had no choice but to switch out a box with the IBH left at Danger), and I know drivers who have had to pass IBHs at danger on their own authority too.

 

[Tio Terry]

The probability of a modern control centre (not just ROC's but ECR's as well) finding itself in a close down position is pretty remote. The buildings are fire compartmentalised such that each control room is a 2hr fire resistant room, cabling is LSZH, Air con systems shut down or are designed to extract any smoke and not circulate it. There are two separate Grid power supplies with either a diesel generator or Traction derived third supply - no traction supply means that not very many trains can run in most areas!

With two hours before enforced evacuation that should be enough to bring services to a controlled stop with the minimum of risk.

 

[tygar2]

The probability of a modern control centre (not just ROC's but ECR's as well) finding itself in a close down position is pretty remote. The buildings are fire compartmentalised such that each control room is a 2hr fire resistant room, cabling is LSZH, Air con systems shut down or are designed to extract any smoke and not circulate it. There are two separate Grid power supplies with either a diesel generator or Traction derived third supply - no traction supply means that not very many trains can run in most areas!

With two hours before enforced evacuation that should be enough to bring services to a controlled stop with the minimum of risk.

Wow...that was very educational...thank you!

 

[D Foster]

It seems to me that some degree of movement management could be maintained using pencil and paper (to record what is where - which is critical) and assorted telephones. There is also the practical possibility of the drivers of trains on one line that have been contacted stopping and talking to other drivers to relay information about who to talk to on what numbers. (Assuming that GSM isn't in use for some reason).
About 2009(ish) when the storms flooded Cumbria Network Rail pulled out a large stock of satellite phones and worked with those. Although we should recall that quite a lot of phones are still on landlines - SPTs for a start. So long as these aren't routed through some form of digital or link system (e.g. microwave) it should be possible to use them. (Depending, of course, on whether the signalling centre is just "down" or fully evacuated.

Over all though it seems to me that the only thing that would truly wipe out the whole system - all phones included - would be massive sun-spot activity or a nuclear blast - in either of which cases I don't think many people would be worrying about carefully moving trains...

In the past I have been around when Guildford power box lost all its wiring - a 12 car train shunted into a 4 car siding and wiped out the location boxes that had been neatly placed behind the buffer stops... After a pause everyone just worked with pen and paper using what phone lines remained available - those that didn't go through the location boxes that had been squashed.

 

[LAX54]

The rule book still implies that we can pass these if the box is closed. Surely if the box is on fire and the staff are all outside at the assembly point, it's closed!?

But ..in the extreme.. the fire could affect the wiring, the relays, electrnic links, and move points / cause COA's without anyone knowing !

It is rare for a ROC /IECC / PSBN to close completely these days, and if it does, it would not be for very long, albeit enough to decimate the service, depending on the type of alarm sounding in the box, most trains would be brought to a stand in the Station,

What is more common is for Signalling System failures, and even more so now its computer controlled, (more to go wrong !) NX is a bit more stable, and old school almost, but with such a loss, trains would stop, and stay stopped, until its either fixed, or a method of working in place, now if indications are lost, any points within a route that the train wishes to use will have to be clipped and locked manually, this is what takes time
( If the override( if fitted) system works, thats fine, trains can carry on albeit with reduced routes available, so may have to terminate short.

 

[najaB]

Theory only. It cannot be done in practice.

Not even in theory. Plans to have ROCs backing each other up were dropped at least 7 years ago.

If the signalling equipment is decoupled from the control equipment and they speak through a standardised communication interface (which is my understanding) why isn't it possible for any workstation in any ROC to replicate any workstation in any other ROC?

 

[Tomnick]

If the signalling equipment is decoupled from the control equipment and they speak through a standardised communication interface (which is my understanding) why isn't it possible for any workstation in any ROC to replicate any workstation in any other ROC?

My (relatively limited) understanding is that the whole system would have to be fully tested every time the communications channels between control system and signalling equipment were, so to speak, 'rerouted'. That, plus the fact that most workstations have some bespoke equipment - CCTV crossings, monitors and controls, emergency overrides and that sort of thing - that can't easily be replicated elsewhere. I'm sure that it could be done fairly quickly - probably days rather than hours - if necessary, though. Hopefully a more enlightened S&T chap will be along shortly!

The big short-term problem, though, is that it isn't practical to ensure that there are suitable competent staff (bearing in mind the specific competency required for each workstation, and the frequency with which each must be worked to retain that competence - refer to the recent level crossing collision on the Ely - Norwich line), and in sufficient numbers, to allow any ROC's workload to be quickly taken over by another.

 

[najaB]

The big short-term problem, though, is that it isn't practical to ensure that there are suitable competent staff (bearing in mind the specific competency required for each workstation, and the frequency with which each must be worked to retain that competence - refer to the recent level crossing collision on the Ely - Norwich line), and in sufficient numbers, to allow any ROC's workload to be quickly taken over by another.

I understand that staff knowledge would be the limiting factor, I was just wondering about the hardware side of things.

I was thinking about a situation where a workstation wasn't going to be available for an extended period - what are the thing(s) which stop that workstation being replicated somewhere else so that the people who normally work it could continue to work it in a different location (be it a different floor of the same building, or a completely different facility).

 

[edwin_m]

If the signalling equipment is decoupled from the control equipment and they speak through a standardised communication interface (which is my understanding) why isn't it possible for any workstation in any ROC to replicate any workstation in any other ROC?

Certainly with the original SSIs and IECCs I was familiar with it would have been possible to re-configure a signalling centre with the data for a different signalling centre and with an appropriate comms link to the area it was configured to control it would probably talk to the trackside equipment. I can think of several reasons why it's unlikely in practice:

(1) Any such re-configuration would require huge amounts of testing to comply with standards. By the time it was all tested the original centre would probably have been restored.

(2) Although the screens look the same, the ROC equipment comes from several different suppliers and it's unlikely that A's equipment could be made to run as B's control centre without quite a lot of work to re-configure the data.

(3) With the 1980s/90s vintage IECCs the integration of the SPT concentrator was limited to mounting it in the workstation next to the IECC screens. I don't know how easy that would have been to re-configure to another site, nor if integration is any better today.

(4) All the ROC equipment is likely to be doing something already - there is unlikely to be a spare one knocking around to take over unless you're lucky enough that the major failure happens when a new centre has been completed but not brought into use.

 

[LAX54]

If the signalling equipment is decoupled from the control equipment and they speak through a standardised communication interface (which is my understanding) why isn't it possible for any workstation in any ROC to replicate any workstation in any other ROC?

It could be done in theory, but with CCTV links and equipment, phones . SPT's GSM-R terminals, it would be a logistical nightmare, you would also need quite a bit of unused space in a ROC to house the workstations for the other area.
Add to that the requiremenst for a Signaller to work a location at least once every six months to keep his / her compentency, which would also create problems. To fire the w/stn up from cold and tested would take some considerable time testing routes and locking, looking at 2 or 3 days intensive testing, and then find a cable fault somewhere between say Three Bridges and York ! then a 4 to hour refresher for the Signaller, overseen by a Manager who can also do the same job, so no doubts has to be brought in from that area.
Duplicating all the wiring that normally runs Box A, to run to Box B as well, all adds up to being uneconomical.
Its easier to live with the downtime of a failure in the original area.

 

[najaB]

Thanks Tomnick, swills and edwin_m - sounds like we aren't there yet (unfortunately). Maybe ETCS will allow the complete decoupling of control and signalling infrastructure.

 

[GW43125]

In the past I have been around when Guildford power box lost all its wiring - a 12 car train shunted into a 4 car siding and wiped out the location boxes that had been neatly placed behind the buffer stops... After a pause everyone just worked with pen and paper using what phone lines remained available - those that didn't go through the location boxes that had been squashed.

Ah yes, the "emergency train describer"

 

[LAX54]

Thanks Tomnick, swills and edwin_m - sounds like we aren't there yet (unfortunately). Maybe ETCS will allow the complete decoupling of control and signalling infrastructure.

I have my doubts ETCS will make much difference, whereas the way the Railway works is quite 'locked down' and we have such strict things we can do, and cannot do, is the main reason that our system is still the safest in the World

All eggs in one basket is fine whilst it all works OK, it is when there is a glitch in the system it affects a much wider area.

 

[Bald Rick]

If the signalling equipment is decoupled from the control equipment and they speak through a standardised communication interface (which is my understanding) why isn't it possible for any workstation in any ROC to replicate any workstation in any other ROC?

It's the requirement to do 'through testing' that is the technical issue. Every function must be tested from the control equipment through to the ground equipment via the interlocking; every aspect in every signal, every set of points, every track circuit and every route. To put that into context, a relatively complex area will take 2-3 days to test, with no trains running. If you have a duplicate workstation, you double that time. That's an extra 2-3 days guaranteed disruption of no trains to offset what might be a once every 5-10 year problem for a couple of hours. And even then, in the event that the control equipment in a ROC goes down completely, it is likely that the interlockings would too, so you couldn't control the kit remotely anyway.

 

[edwin_m]

It's the requirement to do 'through testing' that is the technical issue. Every function must be tested from the control equipment through to the ground equipment via the interlocking; every aspect in every signal, every set of points, every track circuit and every route. To put that into context, a relatively complex area will take 2-3 days to test, with no trains running. If you have a duplicate workstation, you double that time. That's an extra 2-3 days guaranteed disruption of no trains to offset what might be a once every 5-10 year problem for a couple of hours. And even then, in the event that the control equipment in a ROC goes down completely, it is likely that the interlockings would too, so you couldn't control the kit remotely anyway.

Yes, that requirement would have to go if backup ROCs were ever to be possible. I think it could be done without compromising safety, for example by byte-by-byte verification that the replacement workstation/interlocking was identically configured, and it would have lots of other benefits such as reducing commissioning times. But the signalling profession is very conservative - understandable in a lot of ways - so it's unlikely to happen in an environment where safety or the perception thereof trumps any other consideration.

 

[najaB]

Yes, that requirement would have to go if backup ROCs were ever to be possible. I think it could be done without compromising safety...

It would be possible if there was full separation of the on-ground signalling equipment from the control equipment.

So there would be a box on the ground that interfaces to the actual track circuits, points, signals, etc. and the control equipment i.e. the workstation would 'speak' with that box via a communication link.

If that were the case, you would only need to prove that the workstation can speak to the box - it wouldn't really matter where the workstation was physically located.

 

[edwin_m]

It would be possible if there was full separation of the on-ground signalling equipment from the control equipment.

So there would be a box on the ground that interfaces to the actual track circuits, points, signals, etc. and the control equipment i.e. the workstation would 'speak' with that box via a communication link.

If that were the case, you would only need to prove that the workstation can speak to the box - it wouldn't really matter where the workstation was physically located.

That's pretty much what happens between the interlockings and the trackside. SSI has a serial data bus to up to 63 "trackside functional modules" each with 8 inputs and outputs. The data is encoded to ensure that any corrupted messages are ignored and also that the TFMs configured for one interlocking won't function if connected to a different one. The data stream can be passed over telecom networks between the interlocking and the trackside. More modern computer-based interlockings sometimes still use SSI TFMs and I'm not sure of the details if they use something newer but I guess it would have similar measures to ensure integrity.

 

[Sunset route]

Still doesn't solve the scale of the logistics, TBROC has a provisional floor plan for 37 signalling control desks which take up the complete floor space on the opps floor. Then there is the next floor which is 2/3rds Contol both NR and TOC 1/3rd ECRO desks. Plus all the side offices and support functions. There is not room in the current plans to get the allocated signalboxes/centres in to this building let alone providing duplicated desks to take over any of the fringe ROCs such as Bassingstoke, Didcot, Derby, Rugby, York, Romford or Gilllingham and I should imagine vice-versa.

If you had to shut a completed ROC when fully staffed say like TBROC In years to come where could you transfer it in its entirety, that's must be at least 600 staff including the best part of 200 trained signallers and ECRO Staff.

 

[najaB]

lIf you had to shut a completed ROC when fully staffed say like TBROC In years to come where could you transfer it in its entirety, that's must be at least 600 staff including the best part of 200 trained signallers and ECRO Staff.

I fully appreciate the logistical and staffing issues. I was more thinking about the technological side of things.

 

[Sanatogen]

Passable/non passable signals don't exist anymore, in the sense that new schemes won't have them.

To the OP. Response staff would go out and manually man crossings and points. And using things like pen, paper and magnets trains could very slowly be moved on. Although it's not something I would want to be involved in!


Sent from my iPhone using Tapatalk

I had to do it once after Willesden Suburban moved over to westcad and the whole system just shut down (literally staring at blank screens), we worked with a magnetic board and first established where every train was before platforming what we could without point securing (doubling up in a couple of platforms) and finally having junctions secured for straight line running.

Went on for hours and was probably one of the more challenging days I've ever had.

 

[Sunset route]

I fully appreciate the logistical and staffing issues. I was more thinking about the technological side of things.

And my point I was trying to make was, where were you going to find the 37 spare workstations in neighboring ROCs to even start the technical issue reconfiguring them to take over another ROC. Your going to need to have all hardware in place just gathering dust, with software that will need to updated regularly to allow any ROC to take over any ROC unless you limit it to neighbouring pairs?

 

[najaB]

And my point I was trying to make was...

Again, I get that point: logistically it isn't feasible for one ROC to do the work of two.

Thank you for your answer, but that isn't the question I was asking. The question I asked, and others answered, was what is involved in one workstation being replicated by another.

 

[Llanigraham]

Again, I get that point: logistically it isn't feasible for one ROC to do the work of two.

Thank you for your answer, but that isn't the question I was asking. The question I asked, and others answered, was what is involved in one workstation being replicated by another.

As I understand it, the same as has been answered about ROC's but on a smaller scale.

 

[Sunset route]

Again, I get that point: logistically it isn't feasible for one ROC to do the work of two.

Thank you for your answer, but that isn't the question I was asking. The question I asked, and others answered, was what is involved in one workstation being replicated by another.

But I can't see any reason why you would only replicate only one workstation from ROC "a" in 200miles away in ROC "b". But the principle of if you move one or a whole load is going to b the same. Just that any spare wostaion in theory must be able to takeover any other in the country and that's a question for the S&T boffins.

 

[carriageline]

I quizzed the S&T commissioning guys at ours about doing so.

The short answer, with the railway set up as it is, it's impossible.

The long answer, as previously discussed, is commissioning and testing would be a nightmare. Then there is things like how the equipment is set up, and what it expects to receive commands from, and where that equipment is physically located. Of course, nothing that is insurmountable, but it still applies and is stopping us.

It's probably cheaper for NR to pay for the odd evacuation/fault/failure than to provide all this additional equipment, and whatever else they need to make it feasible to operate.


Sent from my iPhone using Tapatalk

 

[nom de guerre]

The last time (and possibly the only time)a full real-life evacuation of ROC signallers started to take place, which was a few months ago at Three Bridges and mentioned on these forums at the time, the incident was brief, occurred during the close of service...

Not the only time and possibly not the most recent time, either: my local ROC has been fully evacuated during the last six months.

A derelict pub in a neighbouring(!) town was set alight and some of the smoke was ingested by the building's air vents, triggering the fire alarms.

Luckily the service was winding down, so there weren't too many trains about. The signallers on duty platformed everything and set up suitable GSM-R broadcasts before vacating the premises. IIRC everyone was back inside within 30 minutes.

 

[najaB]

The long answer, as previously discussed, is commissioning and testing would be a nightmare. Then there is things like how the equipment is set up, and what it expects to receive commands from, and where that equipment is physically located. Of course, nothing that is insurmountable, but it still applies and is stopping us.

From an IT systems point of view that seems a very inefficient way to do things. I fully understand that historically that was the best way to do things, but I was hoping that more modern systems would be set up differently.

Early computer systems were either monolithic or consisted of very tightly coupled component sub-systems. It's easier (and possibly faster) to code this type of system since you don't need to make it play nicely with anyone else's code, but it makes testing and upgrades a nightmare as you have to do complete end-to-end testing.

Modern systems use what some people call the 'Unix philosophy' - write small modules, make them do one thing well, assume that they are going to be reused as part of a larger system so make sure they play well and test the heck out of them. It takes slightly longer to write this way, but it does mean that you don't need to do complete end to end testing when there are upgrades - you just need to make sure that the component still behaves the same way for the same input. This saves a *ton* of money over the lifetime of the system.

Anyway, enough for the IT lecture, thanks for the answers.

 

[SpacePhoenix]

Don't modern signalling centres do the interlocking in software rather than hardware? (Can't remember where I read about the interlocking being done in software)

 

[Joseph_Locke]

Don't modern signalling centres do the interlocking in software rather than hardware? (Can't remember where I read about the interlocking being done in software)

Yes, that's why it's call CBI (computer-based interlocking). However, not all interlockings are new or even in the ROCs; the ROC technology can remotely control SSI (Solid State Interlocking) which is ZX81-era CBI and older RI (relay interlocking) technology via protocol convertors and remote data links.