Rugby ROC Evacuated - 18/09/18 Evening

So it's finally happened... all signallers evacuated from a Rail Operating Centre due to an actual fire/smoke which can't be contained all that quickly.

"Luckily" (on the scale of it) tonight's incident is Rugby's turn, which currently has diversionary routes around at least part of its present area of control, meaning that some WCML services can still move, albeit with delays. VTWC departures are, however, gradually being suspended at Euston as the fallout is realised.

Several services have been stranded or moving extremely slowly for quite some time now, and it looks like there was vastly insufficient time to make sure all services had reached platforms. In some cases, queues of trains have been stood signal-to-signal at locations away from stations. From my experience, design of such buildings means the signallers and other crucial safety staff are kept on duty for as long as possible within the tolerances of the evacuation system, with gradual warnings about how soon they need to go... so to have trains stood away from platforms indefinitely means the evacuation process can't have been ideal in this case.

As for some passenger info - various buses are now running in the Stafford/Stoke area at least, trying to connect diversionary routes and stations presently without a service.

Would the automatic signalling system still operate during an evacuation?

Also, is it called ROC because I thought it was SCC?

E_Reeves said:

Would the automatic signalling system still operate during an evacuation?

Click to expand...

Apparently it's the server room which is on fire, so I don't think so.

Peter Mugridge said:

Apparently it's the server room which is on fire, so I don't think so.

Click to expand...

Even if it is, the queue will stop once the front train reaches a signal that isn't automatic.

E_Reeves said:

Would the automatic signalling system still operate during an evacuation?

Also, is it called ROC because I thought it was SCC?

Click to expand...

There's a limit to what any automatic signalling can do, especially without anyone to oversee any incidents during its operation. Automatic signals themselves may well work fine, but they are limited in scope for movements available... for example, into complex stations or over junctions!

It's the ROC. Rugby ROC has not yet fully taken over everything, including not yet controlling the Rugby station area itself, perhaps a little ironically! But that's good, really, since that means the WCML isn't completely shut tonight.

The ROC is evacuated which involves the Stafford workstation.
The SCC which controls the bits further south seems to be okay at present.

(I only just found out the difference just now from the reports on this!)

Peter Mugridge said:

Apparently it's the server room which is on fire, so I don't think so.

Click to expand...

The server room acocomodates the non vital IT systems (desktop PC network etc..) rather than the actual interlocking, which is located elsewhere in the building.

Highlandspring said:

The server room acocomodates the non vital IT systems (desktop PC network etc..) rather than the actual interlocking, which is located elsewhere in the building.

Click to expand...

A detail I'm sure Network Rail won't be keen to talk about: Just how geographically redundant are interlockings? Are they active-active, active-passive or single point of failure?

Stafford appears to be the only station at which no services at all are calling. Some very lengthy delays are accruing on both VT and XC.

Looks like the ROC staff are doing what they can from other locations.

VTWC tickets are OK on Chiltern and EMT.
LNR tickets are OK on VTWC, Chiltern, Merseyrail and ATW.
... (for what it's worth at this time of the day!)

tsr said:

Looks like the ROC staff are doing what they can from other locations.

Click to expand...

Can controlling computer terminals be fired up at other ROCs to control the affected areas while the offending ROC is still evacuated?

Mag_seven said:

Can controlling computer terminals be fired up at other ROCs to control the affected areas while the offending ROC is still evacuated?

Click to expand...

Even if they could, would it be safe for staff to control an area they don't know? Could possibly end up with fast services on slow lines, or wrong-routing.

Mag_seven said:

Can controlling computer terminals be fired up at other ROCs to control the affected areas while the offending ROC is still evacuated?

Click to expand...

There are two different functions in a ROC; signalling workstations operate the signalling equipment and can't be relocated elsewhere but the Operations Control can use an alternate contingency location with access to suitable PCs and phones. There's not really much that can be said publically about the resilience planning and fall back capability

Anyway, the all clear was given at 2315 and everyone is now back inside.

SErvices now moving from Stafford - 1U47 to Crewe currently 153 down, 1A71 to Euston currently 155 down.

E_Reeves said:

Would the automatic signalling system still operate during an evacuation?

Also, is it called ROC because I thought it was SCC?

Click to expand...

Would still need Signallers there for emergencies etc. Wouldn't be a great scenario for a fatality/derailment to occur without a signaller being present.

marko2 said:

A detail I'm sure Network Rail won't be keen to talk about: Just how geographically redundant are interlockings? Are they active-active, active-passive or single point of failure?

Click to expand...

The more I think about it, why would computer-based interlockings be in a ROC at all when they can effectively be anywhere accessible on the FTN? Tier IV commercial data centres are phenomenally robust - and systems and spacial redundancy is far more easily achieved at scale.

Incidentally there's only one signalling workstation (Stafford, commissioned 01/09/15) at present in Rugby ROC. There are five workstations in the SCC across the road.

I was under the impression there was overlap between the new signalling centres so in the event of one going down for whatever reason the nearest ones could step in and take over. Is this right or have I misunderstood?

Would be an improvement over the present TBROC service.

Peter Mugridge said:

Apparently it's the server room which is on fire, so I don't think so.

Click to expand...

Oh dear. Does the server room not have an automatic gas or synthetic chemical fire suppression system? Which should be standard spec in any mission-critical building.

If not, did some beancounter decide it could just be left out or 'value engineered' (a term which usually has no relation to either value or engineering)

londiscape said:

Oh dear. Does the server room not have an automatic gas or synthetic chemical fire suppression system? Which should be standard spec in any mission-critical building.

If not, did some beancounter decide it could just be left out or 'value engineered' (a term which usually has no relation to either value or engineering)

Click to expand...

Yes it does have an automatic fire suppression system fitted

Eyersey468 said:

I was under the impression there was overlap between the new signalling centres so in the event of one going down for whatever reason the nearest ones could step in and take over. Is this right or have I misunderstood?

Click to expand...

That was just the hype on the glossy brochure until the reality kicked in.

If every signalling workstation in every ROC was maxed out to the amount of railway and work each signaller is controlling, all at 100% with normal running. Then if every ROC is designed for the maximum amount of workstations just to control its area based on the above. So now we are where the dream hits reality as if the signaller is working at 100% capacity under normal running, then what happens in the times of failure. There is now more workload than a idvidual can safely handle, you can’t pass some workload to their colleagues as they are also running at 100% under normal running plus working extra from the fallout from your problem. So now the areas of control has shrunk so each signallers is working around 70% or capacity to allow for working in times of disruption. This has had the knock on effect that you now need more workstations in buildings that weren’t designed to house that many, hence the mini or sub ROC a concept has been touted.

Also if ROCs are full of workstations controlling their own part of the railway network, where do you put say 25% extra workstations to control say the largest ROC (York I beleive) into say 4 other ROCs taking equal share of the workload. There are at least 7 types (or sub types) of signalling workstations that I’m aware of out there from the various signalling manufacturers each which requires training on, all local rules per each location/workstation is unique to the location. The generic rule book does not cover everything. If you do find space to instal these extra workstations, will they mirror the same type of the ROC who’s area is being taken over or will it be of the same type already installed in the ROC which is taking the extra workload and does that pose any signalling control risk. When do you put this over capacity into each ROC, in stages keeping pace with the migration project or once its finally completed in many decades time.

But the big issue is who will man theses dust covered workstations, will it be the staff from the effected ROC after being put up in local hotels and how long will the railway shutdown be until they can temporarily relocate. Or are they going to go to the government and say that despite the ROC concept being partially about signalling staff reducuctions that they will need to employ a pool of high grade signallers to sit aound in ROCs basically doing nothing to act as standby contingency just in case there is a rare occurrence of losing a complete ROC. Who will maintain their competency and how often would they get to actually control real trains to keep their skills and knowledge up?

Makes change from a funeral service being held for the toaster adjacent to the waste paladins

E_Reeves said:

Would the automatic signalling system still operate during an evacuation?

Also, is it called ROC because I thought it was SCC?

Click to expand...

Rugby has two Signalling Centres about 300 yards apart.

Rugby SCC right next to rugby station, and Rugby ROC which is a little way up the yard.

ROCS are designed to keep operational staff insitu as long as possible in the event of fire to allow signallers to stop trains in a safe manner.

However, I’m not changing my luck on what “maybe” under the floor I’m working on.

Auto signalling/ARS could in theory run the job whilst the signallers are evacuated, however a workstation can not be left unattended whilst trains are running due to the 101 things that coils go wrong and require action.

Mag_seven said:

Can controlling computer terminals be fired up at other ROCs to control the affected areas while the offending ROC is still evacuated?

Click to expand...

That was the big promise when the magical ROC concept was planned, if Rubgy went down then say West Midlands could take over at the flick of a switch.

However, that would require vast banks of workstations and competant staff ready and waiting.

Tom Quinne said:

Rugby has two Signalling Centres about 300 yards apart.

Rugby SCC right next to rugby station, and Rugby ROC which is a little way up the yard.

ROCS are designed to keep operational staff insitu as long as possible in the event of fire to allow signallers to stop trains in a safe manner.

However, I’m not changing my luck on what “maybe” under the floor I’m working on.

Auto signalling/ARS could in theory run the job whilst the signallers are evacuated, however a workstation can not be left unattended whilst trains are running due to the 101 things that coils go wrong and require action.

Click to expand...

Ah so the SCC is the very bland building that has Rugby SCC on the side and is the ROC the large green building with Network Rail on it?

Eyersey468 said:

I was under the impression there was overlap between the new signalling centres so in the event of one going down for whatever reason the nearest ones could step in and take over. Is this right or have I misunderstood?

Click to expand...

That's what I thought..

So, hypothetically - could something like the following happen in the future....

When Peterborough box finally goes into York ROC, it will (eventually) be in a triangle of ROC's (York, EMCC at Derby, and Colchester/Romford)

So, let's say York ROC has a SERIOUS fault at the Southern end, would they "switch in" at EMCC or Colchester/Romford - or could they go into (what is now) the CURRENT box here at Peterborough and control things from there...

Just a thought....

E_Reeves said:

Ah so the SCC is the very bland building that has Rugby SCC on the side and is the ROC the large green building with Network Rail on it?

Click to expand...

Yep that’s the one

Stampy said:

That's what I thought..

So, hypothetically - could something like the following happen in the future....

When Peterborough box finally goes into York ROC, it will (eventually) be in a triangle of ROC's (York, EMCC at Derby, and Colchester/Romford)

So, let's say York ROC has a SERIOUS fault at the Southern end, would they "switch in" at EMCC or Colchester/Romford - or could they go into (what is now) the CURRENT box here at Peterborough and control things from there...

Just a thought....

Click to expand...

No, the piggy bank is empty many Resignalling schemes which have gone to far go pull the plug on will be no frills like for like, so any notion of switching in another ROC is long long dead.

Sunset route said:

That was just the hype on the glossy brochure until the reality kicked in.

Click to expand...

oops!
ALL critical infrastructure needs work-arounds built in.
just think of the internet and your packets of data being transported around.

If a node/hub goes down there is a network of other hubs/nodes that are in place to redirect traffic.

I would suggest that the rail industry needs to get up to speed on this pronto, otherwise they will find full automation upon them rather quickly