All Northern self-service ticket machines off line 13/7/2021

[Dai Corner]

As I said, the ticket machines themselves are fine, it's the management system that's the issue here...

It's a malware attack affecting the 'management system' then?

== Doublepost prevention - post automatically merged: 18 Jul 2021 ==

Has it affected other TOCs with the same machines e.g. Chiltern? If not, what is different?

Chiltern's 'management system ' wasn't targeted or is better protected?

 

[scrapy]

Could an individual TOC have such specialist activities entirely in house, ?

Doubt it, never suggested they could.

 

[johntea]

A malware attack is bad news, you can't just restore from a backup as otherwise you just end up back at square one! (As the backup may still contain the entry route for the hackers)

If you don't know with absolute confidence how they got in and how to close that door the only option is to basically build the system up again...from scratch!

May also suggest why they're trying to turn off ticket vending machines to prevent the spread to those

 

[Carlisle]

Doubt it, never suggested they could.

Your post suggested either outsourcing or poor choice of contractor could be a significant contributor to the problems under discussion here.

 

[Djgr]

Could an individual TOC have such specialist activities entirely in house, ?

Could the railway industry though? Should the industry being outsourcing this to some Tom, Dick and Harry?

 

[tarq]

I’m rather hoping we get a consistent approach to new TVMs under GBR.

 

[py_megapixel]

I’m rather hoping we get a consistent approach to new TVMs under GBR.

My gut feeling is that they'll be consistent, but utter s**t...

 

[Steddenm]

Can we go back to the Quickfare machines? At least they were standalone.


(Image shows an Ascom self service ticket machine known as Quickfare and provided by British Rail)

 

[MountVrmac]

It's annoying when the TVMs are not working at unstaffed stations, and it's inconvenient when the guard doesn't pass through the train to sell you a ticket; though I can understand they may have other duties, or don't want to slowly pass through so many non-mask wearing customers, even before Monday 19th July when the law changed.

But what made me furious on Saturday 17th July was getting into Leeds on a sunny weekend day with many leisure travellers and joining a VERY long queue platform-side to by tickets from the one person at the Ticket kiosk. They were eventually joined by a second person, but there were five revenue protection officers, presumably hoping to catch someone who had travelled from a station with ticket machines provided by Cross Country or LNER.

I'm all for catching genuine fare dodgers, but the optics of the RPOs all standing around, while only two people were selling tickets (especially egregious when they have a colleague sitting in the ticket kiosk but not working) just beggared belief.
The TVM issue had been known of since Tuesday, and the good weather was also a known factor - I simply can't understand why a battalion of staff armed with ticket machines wasn't deployed on what was going to be a very busy period. The average passenger doesn't care about the demarcations between Network Rail, Northern Railway or the RPOs.

 

[Djgr]

It's annoying when the TVMs are not working at unstaffed stations, and it's inconvenient when the guard doesn't pass through the train to sell you a ticket; though I can understand they may have other duties, or don't want to slowly pass through so many non-mask wearing customers, even before Monday 19th July when the law changed.

But what made me furious on Saturday 17th July was getting into Leeds on a sunny weekend day with many leisure travellers and joining a VERY long queue platform-side to by tickets from the one person at the Ticket kiosk. They were eventually joined by a second person, but there were five revenue protection officers, presumably hoping to catch someone who had travelled from a station with ticket machines provided by Cross Country or LNER.

I'm all for catching genuine fare dodgers, but the optics of the RPOs all standing around, while only two people were selling tickets (especially egregious when they have a colleague sitting in the ticket kiosk but not working) just beggared belief.
The TVM issue had been known of since Tuesday, and the good weather was also a known factor - I simply can't understand why a battalion of staff armed with ticket machines wasn't deployed on what was going to be a very busy period. The average passenger doesn't care about the demarcations between Network Rail, Northern Railway or the RPOs.

Because they really don't care about passengers?

 

[Bletchleyite]

My gut feeling is that they'll be consistent, but utter s**t...

There's very little wrong with the Scheidt & Bachmann machines, so if they could pick those...

 

[py_megapixel]

There's very little wrong with the Scheidt & Bachmann machines, so if they could pick those...

Are those the ones GWR use? If so, I find them OK, but they could really do with the ability to issue tickets for other dates and stations, and also the screens need to be readable in a brightly lit platform.

 

[Bletchleyite]

Are those the ones GWR use? If so, I find them OK, but they could really do with the ability to issue tickets for other dates and stations, and also the screens need to be readable in a brightly lit platform.

GWR have a mix. They are the small-screen ones used by WMT among others. They can issue for other dates and stations, and the smaller screen is very readable compared with the large ones.

 

[Wolfie]

Of course I would like it fixed, my initial comment was tongue in cheek, it has sprung a lot of defensive responses.

As I said before it should have been tested before roll out or somebody said above " Test twice, deploy once"

There is clear failure in the "fixes"

I always say "Prior Planning and Preparation Prevents Poor Performance"

Re your last sentence: you ex-military perchance? Their version includes another P, "pee" to be polite, before poor....

 

[Bletchleyite]

Re your last sentence: you ex-military perchance? Their version includes another P, "pee" to be polite, before poor....

I think there's a game of how many Ps you can get..."proper" on the start too!

 

[Wolfie]

I think there's a game of how many Ps you can get..."proper" on the start too!

Haha... I was never in the military but worked for MOD for years. I had a crusty old Warrant Officer give me the SP in my very early days...

 

[dustmite]

The BBC's now running a story on this:

Northern's ticket machines hit by ransomware cyber attack

Northern rail's new self-service machines were installed at 420 stations two months ago.

www.bbc.co.uk

 

[TheGarner]

Now being reported on the BBC that its due to a ransomware attack: https://www.bbc.co.uk/news/uk-england-57892711

 

[Vespa]

Re your last sentence: you ex-military perchance? Their version includes another P, "pee" to be polite, before poor....

Maybe

The 6 P's is a good mantra to live by.

The 7 P's might be a bit harsh to ears of civvies

 

[js1000]

I would hope penalty fares have been paused until the machines are back in use. Massive queue for ticket kiosk on platform 11 at Manchester Piccadilly today.

 

[MrTurbostar170]

Revenue officers were at Preston when I arrived earlier today.

 

[scrapy]

Your post suggested either outsourcing or poor choice of contractor could be a significant contributor to the problems under discussion here.

Poor choice of contractor possibly. They don't have control over what can or cannot be done in house. Obviously the current contractor is falling well short.

== Doublepost prevention - post automatically merged: 19 Jul 2021 ==

I would hope penalty fares have been paused until the machines are back in use. Massive queue for ticket kiosk on platform 11 at Manchester Piccadilly today.

They have where appropriate.

 

[jon0844]

I read that in Sweden an entire chain of supermarket had to close due to a problem with the company providing services for the tills/payment processing. A company that wasn't actually contracted to the supermarket, but the third-party.

It's unrealistic for a TOC with a 7 year franchise to set up its own system, so it was always likely certain services would be outsourced. Small businesses don't create their own bank to take cards - they employ the services of another company to do it, and are then at the mercy of that company.

Just as affects online businesses who might find their server goes offline, or cloud-based services are unavailable for a period of time.

I think such issues are only going to get worse as time gets on, and people begin to realise just how complex some systems are. It reminds me of the process I need to go through to renew my brown bin (garden waste) permit via my local council. You start on the council website, then are immediately transferred to a third party site that manages the permit and database, then on to another site (worldpay) to make the payment and then back to the third-party site and then finally back to the council webpage. So many chances for something to go wrong, and a nightmare for the person(s) impacted to get it sorted as one company passes the buck to another.

 

[Skie]

Telling thing here is how long they’ve been down for. Ransomware attacks are only effective if you don’t have a working backup solution. If you do, it’s trivial to just wipe the affected machines and redeploy via the backup (and build scripts).

If not, then you’ve got to rebuild everything or pay the ransom.

 

[jon0844]

Telling thing here is how long they’ve been down for. Ransomware attacks are only effective if you don’t have a working backup solution. If you do, it’s trivial to just wipe the affected machines and redeploy via the backup (and build scripts).

If not, then you’ve got to rebuild everything or pay the ransom.

I was quite surprised to learn from people I know in the IT world that a lot of businesses DO pay (or an insurance company does). Some companies even rely on insurance to the point where they'll invest less in security measures because there's 'no need' to spend their money for a problem that they can claim against.

I'm told insurers are rapidly seeking to change policies, which might focus a few minds in the future.

To make it clear, I'm not suggesting any of what I've been told is related to this issue in any way whatsoever.

 

[skyhigh]

Telling thing here is how long they’ve been down for. Ransomware attacks are only effective if you don’t have a working backup solution. If you do, it’s trivial to just wipe the affected machines and redeploy via the backup (and build scripts).

If not, then you’ve got to rebuild everything or pay the ransom.

Although there's little to no point putting exactly what you had to start with back on before you're confident the vulnerability is fixed

 

[Wolfie]

Although there's little to no point putting exactly what you had to start with back on before you're confident the vulnerability is fixed

Absolutely.

 

[syorksdeano]

How long did it take Merseyrail to sort out when they were attacked in April this year?

And more importantly how did this incident happen?

 

[Skie]

Although there's little to no point putting exactly what you had to start with back on before you're confident the vulnerability is fixed

Well you put it back but sandboxed so nobody can access it but your team. And preferably review all access and logs just in case someone slipped something in months before the attack.

 

[deep south]

This is now been ongoing for over a week; this is "somewhat amateur hour" at the very least... a clean rebuild from scratch and thorough testing shouldn't take this long.

I wonder if we have a case here of "it's nearly fixed, just wait a couple of hours.... oh dear, still broken, let's try again" is being repeated. Of course the IT support team has probably been cut to the bone, or outsourced to the cheapest supplier (and has been said, Northern Rail were stuck with the incumbent...) so they may not have sufficient experienced staff to resolve in a timely manner.

if the contract doesn't have penalty clauses then the purchasing team - and senior exec responsible - need a stiff talking to. I wonder if his bonus can be clawed back - if he is still there of course.