RealTimeTrains website

[FGW_DID]

If every post querying ”RTT hiccups” and moaning about certain TOC allocations not showing on a particular day were to be removed from this thread, I doubt it would be past Page 30 (being generous there as well!)

 

[Class15]

That's extremely unfair - as someone else has pointed out, it was the Network Rail data feed
which had a 'meltdown' yesterday, so the issue was completely out of the control of RTT,
as it is in 99% of such cases.

It affected all the 'realtime' sites including Freightmaster, which caused a few of my customers
to ask me what was happening, but as far as I am aware, the only person who thought I was
having a meltdown yesterday was Mrs Freightmaster!



MARK

Indeed, the problems on Railcam went on longer than those on RTT which is most surprising considering the former generally has a much more reliable feed!

 

[Robin Procter]

If every post querying ”RTT hiccups” and moaning about certain TOC allocations not showing on a particular day were to be removed from this thread, I doubt it would be past Page 30 (being generous there as well!)

.... It's through querying (I wasn't moaning, in fact I said quite the contrary) that we all learn how and can be tolerant of how things work. Feedback, both positive and negative is a good thing (in my opinion) and a group internet forum such as this one is a relevant arena to express it.

 

[takno]

Indeed, the problems on Railcam went on longer than those on RTT which is most surprising considering the former generally has a much more reliable feed!

I switched Traksy over to the non-approved feed after about 40 minutes, but that was just a matter of good timing since I was in front of a computer when it went down.. Looks like all the other sites progressively did the same thing over the course of the afternoon. I suppose at some point I'll have to remember to switch back.

 

[Tom]

Well, my plan to have a pleasant, quiet weekend away from the computer and work having a housewarming with family in my new home didn't go so well ...

Let's start with the ads: the issue continued after we were told that it was resolved. To that end, we took the decision late last night to remove the affected units on all platforms. I will be making my views known to our provider on Monday and we will likely be looking at moving away from them due to, frankly, how appalling their service has been on this matter. That has resulted in a significant loss of revenue but better to lose money and have users than have no users and no money. Not that the data feeds did their bit on helping on that one.

And now the bigger issue of the weekend, the data feeds. This was entirely out of RTT's control and numerous other sites were affected. Amusingly, allocations continued to update throughout the mess as they come through a completely different set of feeds.

They started to die between 0930 and 1000 as far as I can make out and latency started slowly creeping up at this time, by 11am they were 8 minutes behind and at midday it was an hour. RTT would have started to show the no data message between 1000 and 1100 - I don't actually internally monitor when it shows as we use separate monitoring to alert us. I was out shopping and at that point it's normally a case that it will just fix itself - as the message on RTT itself states. My message logs show that we just kept receiving the same batch of messages from the same 30-40 second block of 1104 most of the time from around midday through to midnight when they finally died completely.

At about 1230, I decided to migrate RTT over to the internal industry feeds as I currently have no confidence in CACI (NR's supplier) to provide a reliable feed so moving over to the 'unapproved' platform didn't seem wise. Once I have confidence back, I intend to shift back because failures need high visibility to get attention from the right parts. For now, failures will probably not affect RTT as we haven't had a single instance of data loss on the internal feeds in the 2.5 years we've had access to them.

Now, to cover off some other points:

It's interesting that the LNER services now show on RTT as VSTP, rather than WTT, which is what they really are - and is confirmed by the UIDs. Presumably, that's because of how they got there, rather than what they are.

The CAN deletion doesn't remove the VSTP alteration flag in the database at the moment - there's an open bug in our issue tracker to separate VSTP alteration and WTT schedules. However, you can have STP VSTPs running with a normal WTT style UID from time to time, it's not that unusual.

Just a hiccup with the data feeds that happens from time to time. Affected all tracking sites not just RTT.

I think I have a more choice 11 letter word beginning with C for this one.

Indeed, the problems on Railcam went on longer than those on RTT which is most surprising considering the former generally has a much more reliable feed!

We use(d) exactly the same feeds, I suspect RTT is just far more openly hostile about it (and somewhat less tolerant, too) when there are issues with that feed - see https://www.realtimetrains.co.uk/about/status/ for a good example. When it says "last 5 minutes" it means it has received a message that was delivered and timed to be within the last 5 minutes. If it is, say, 22:00 and it is receiving messages from 21:40 then the No Data messages and that page is going to be full of crosses.

 

[Robin Procter]

Well, my plan to have a pleasant, quiet weekend away from the computer and work having a housewarming with family in my new home didn't go so well ...

Let's start with the ads: the issue continued after we were told that it was resolved. To that end, we took the decision late last night to remove the affected units on all platforms. I will be making my views known to our provider on Monday and we will likely be looking at moving away from them due to, frankly, how appalling their service has been on this matter. That has resulted in a significant loss of revenue but better to lose money and have users than have no users and no money. Not that the data feeds did their bit on helping on that one.

And now the bigger issue of the weekend, the data feeds. This was entirely out of RTT's control and numerous other sites were affected. Amusingly, allocations continued to update throughout the mess as they come through a completely different set of feeds.

They started to die between 0930 and 1000 as far as I can make out and latency started slowly creeping up at this time, by 11am they were 8 minutes behind and at midday it was an hour. RTT would have started to show the no data message between 1000 and 1100 - I don't actually internally monitor when it shows as we use separate monitoring to alert us. I was out shopping and at that point it's normally a case that it will just fix itself - as the message on RTT itself states. My message logs show that we just kept receiving the same batch of messages from the same 30-40 second block of 1104 most of the time from around midday through to midnight when they finally died completely.

At about 1230, I decided to migrate RTT over to the internal industry feeds as I currently have no confidence in CACI (NR's supplier) to provide a reliable feed so moving over to the 'unapproved' platform didn't seem wise. Once I have confidence back, I intend to shift back because failures need high visibility to get attention from the right parts. For now, failures will probably not affect RTT as we haven't had a single instance of data loss on the internal feeds in the 2.5 years we've had access to them.

Now, to cover off some other points:

The CAN deletion doesn't remove the VSTP alteration flag in the database at the moment - there's an open bug in our issue tracker to separate VSTP alteration and WTT schedules. However, you can have STP VSTPs running with a normal WTT style UID from time to time, it's not that unusual.


I think I have a more choice 11 letter word beginning with C for this one.


We use(d) exactly the same feeds, I suspect RTT is just far more openly hostile about it (and somewhat less tolerant, too) when there are issues with that feed - see https://www.realtimetrains.co.uk/about/status/ for a good example. When it says "last 5 minutes" it means it has received a message that was delivered and timed to be within the last 5 minutes. If it is, say, 22:00 and it is receiving messages from 21:40 then the No Data messages and that page is going to be full of crosses.

.... Thank you for taking the time to inform us all about the problems and in such detail - I think it helps many of us appreciate the work you do Tom. I have heard by word of mouth that RTT is not your full-time job, so extra kudos to you if that is the case.

On the subject of adverts appearing on your RTT pages, I use adblocker apps on all my devices (mobile and desktop) and so I see no ads, not even on Facebook - Does having adblockers effect any benefits to RTT?

 

[Tom]

No difference to you, less money to keep the site running and keep people employed to me. We get paid per ads that get displayed to you, rather than per click.

 

[Robin Procter]

No difference to you, less money to keep the site running and keep people employed to me. We get paid per ads that get displayed to you, rather than per click.

.... Thanks, clicks vs display is why I asked. I can now enable ads specifically on RTT via the adblocker's control panel so you don't lose money.

 

[ainsworth74]

.... Thanks, clicks vs display is why I asked. I can now enable ads specifically on RTT via the adblocker's control panel so you don't lose money.

Ahem, just to quickly hijack this, the Forum is also funded the same way, hint hint

 

[ABB125]

At about 1230, I decided to migrate RTT over to the internal industry feeds as I currently have no confidence in CACI (NR's supplier) to provide a reliable feed so moving over to the 'unapproved' platform didn't seem wise. Once I have confidence back, I intend to shift back because failures need high visibility to get attention from the right parts. For now, failures will probably not affect RTT as we haven't had a single instance of data loss on the internal feeds in the 2.5 years we've had access to them.

Out of interest, what are the different data feeds that can be used? I've seen various mentions of "unapproved" feeds, why are they unapproved? Thanks!
(I know essentially nothing about data feeds in the real world, as you may be able to tell!)

 

[Tom]

There are three versions of the real-time data feeds.

If I start with the internal feeds from which both of the open feeds are based off. These are the same feeds that go out to all and sundry within the industry, and most industry applications that use signalling data derive themselves from it. These are called TD.net (Train Data over Internet).

The open feeds take the TD.net feed and do reformatting to make it easier to interpret - and they used to obfuscate headcodes but obviously don't anymore, etc. There are two versions of these feeds now - 'approved' and not:

• Approved is intended for already robust implementations that aren't experimenting against the platform and don't cause any issues with the stability from a user end. Essentially, high profile users and already reliable implementations have access to this.
• 'Unapproved' (or I guess public from the domain name - https://publicdatafeeds.networkrail.co.uk/ntrod/welcome) is the same code but runs separately. This is for newer users who are still experimenting, etc, and have less reliable implementations to consume the data which can result in the platform falling over.

This separation was put in place after the unreliability of the old data feeds platform over the years. Entertainingly, the approved platform I think has now been down for longer in a single incident than the public system.

Until 1245 on Saturday, RTT subscribed to the 'approved' platform and pulled all of its train running information from there. It now runs on the internal feeds until further notice.

 

[nw-sparks]

Unfortunately, a crucial member of the Railcam volunteer team (me) wasn't available when the feeds failed and so the issue went on longer than we would have liked. In our case, the only option was to move to the "unapproved" feed, which like Tom, we have little confidence in. Unfortunately, like most websites, Railcam doesn't have the option of switching to the direct Network Rail feeds, so is always at the mercy of CACI's poor performance. In this case, it seems that nobody was around at CACI for the whole of Saturday, so the problem went on for many hours as Railcam, RTT and others scrabbled around to find a way to keep their services going. Very poor indeed CACI.

 

[D6700]

However, you can have STP VSTPs running with a normal WTT style UID from time to time, it's not that unusual.

STP schedules always have a letter, just like WTT schedules do.

VSTP schedules never have a letter.

STP or WTT schedules amended by VSTP (as Varations and Cancellations) always have a letter, which matches that of the underlying STP or WTT schedule.

VSTP schedules further amended by VSTP continue without a letter.

 

[Class15]

Unfortunately, a crucial member of the Railcam volunteer team (me) wasn't available when the feeds failed and so the issue went on longer than we would have liked. In our case, the only option was to move to the "unapproved" feed, which like Tom, we have little confidence in. Unfortunately, like most websites, Railcam doesn't have the option of switching to the direct Network Rail feeds, so is always at the mercy of CACI's poor performance. In this case, it seems that nobody was around at CACI for the whole of Saturday, so the problem went on for many hours as Railcam, RTT and others scrabbled around to find a way to keep their services going. Very poor indeed CACI.

Indeed. Not the fault of you or anyone else at Railcam or RTT!

 

[ABB125]

There are three versions of the real-time data feeds.

If I start with the internal feeds from which both of the open feeds are based off. These are the same feeds that go out to all and sundry within the industry, and most industry applications that use signalling data derive themselves from it. These are called TD.net (Train Data over Internet).

The open feeds take the TD.net feed and do reformatting to make it easier to interpret - and they used to obfuscate headcodes but obviously don't anymore, etc. There are two versions of these feeds now - 'approved' and not:

• Approved is intended for already robust implementations that aren't experimenting against the platform and don't cause any issues with the stability from a user end. Essentially, high profile users and already reliable implementations have access to this.
• 'Unapproved' (or I guess public from the domain name - https://publicdatafeeds.networkrail.co.uk/ntrod/welcome) is the same code but runs separately. This is for newer users who are still experimenting, etc, and have less reliable implementations to consume the data which can result in the platform falling over.

This separation was put in place after the unreliability of the old data feeds platform over the years. Entertainingly, the approved platform I think has now been down for longer in a single incident than the public system.

Until 1245 on Saturday, RTT subscribed to the 'approved' platform and pulled all of its train running information from there. It now runs on the internal feeds until further notice.

Thanks, very interesting

 

[Tom]

STP schedules always have a letter, just like WTT schedules do.

VSTP schedules never have a letter.

STP or WTT schedules amended by VSTP (as Varations and Cancellations) always have a letter, which matches that of the underlying STP or WTT schedule.

VSTP schedules further amended by VSTP continue without a letter.

I agree - but I have observed and do still see from time to time (whether it is right or wrong within the expected system constraints I pass no comment on) what should be a VAR appearing as a STP replacing an otherwise WTT schedule in VSTP land, with the same UID as of it were a VAR.

 

[reduke]

I am assuming this is a mistake

Realtime Trains | 2N67 1854 London Euston to Northampton | 22/07/2023

Real-time train running information for 2N67 1854 departure from London Euston to Northampton on 22/07/2023. From Realtime Trains, an independent source of train running info for Great Britain.

www.realtimetrains.co.uk

 

[DarloRich]

it must be a funny data glitch - bit early for a sleeper!

I wish the sleeper did stop at MK mind - would be very helpful!

 

[Robin Procter]

Ahem, just to quickly hijack this, the Forum is also funded the same way, hint hint

.... I can now see an ad which says "Work Smarter..." and another which says "Reduce your drinking..."

I stopped both working and drinking any alcohol over 20 years ago!

 

[43184spotter]

Real-time trains showing 1S55 being 43301 and showing no second powercar. Anyone know which powercar is the other one? In no world do I see RTT being correct in the fact there’s only 43301

 

[Welshguy1048]

Real-time trains showing 1S55 being 43301 and showing no second powercar. Anyone know which powercar is the other one? In no world do I see RTT being correct in the fact there’s only 43301

You are probably better asking this in TOPS request thread - not here.

 

[FrodshamJnct]

Real-time trains showing 1S55 being 43301 and showing no second powercar. Anyone know which powercar is the other one? In no world do I see RTT being correct in the fact there’s only 43301

With 43239.

 

[43184spotter]

You are probably better asking this in TOPS request thread - not here.

It was the supposed RTT issue/problem/glitch that interested me in it hence why I looked for the RTT forum. If I cared only and purely about 1S55’s allocation, I’d go into the request thread and ask for it

== Doublepost prevention - post automatically merged: 23 Jul 2023 ==

With 43239.

Thank you!

 

[Tom]

RTT shows what’s sent to it by the operator, if the consist is changed in Gemini (or whatever system) after the allocation is sent then they need to “refresh” the allocation - a fairly easy thing for XC - and then RTT will reflect that.

Not really a problem with RTT, but poke XC on the bird app and they can probably fix it.

 

[TT-ONR-NRN]

It would be lovely to see 231 and 197 drawings. Are they due at any point?

 

[158841]

It would be lovely to see 231 and 197 drawings. Are they due at any point?

Was about to ask the same thing!
Be fantastic to see WMT/LNWR join now!

 

[TT-ONR-NRN]

Was about to ask the same thing!
Be fantastic to see WMT/LNWR join now!

Well that’s something else altogether, dependant on WMT of course, whereas I was more referring to drawings for new stock we’ve already got on the site

 

[gazr]

Well that’s something else altogether, dependant on WMT of course, whereas I was more referring to drawings for new stock we’ve already got on the site

I believe this post from a few pages back explains the situation: https://www.railforums.co.uk/threads/realtimetrains-website.189152/page-63#post-6267676

TBH, if I ran RTT, I would probably have chucked it in by now with all these silly requests. You don't think Tom knows what needs doing or what's going on? BE PATIENT.

 

[TT-ONR-NRN]

I believe this post from a few pages back explains the situation: https://www.railforums.co.uk/threads/realtimetrains-website.189152/page-63#post-6267676

TBH, if I ran RTT, I would probably have chucked it in by now with all these silly requests. You don't think Tom knows what needs doing or what's going on? BE PATIENT.

Why be deliberately unpleasant? There is nothing wrong with making a polite request or suggestion, and I’m sure he doesn’t need you making impolite comments or CAPS on his behalf. There’s just no need!

Edit: No, it wasn’t even a request, it was a question!

 

[BRX]

Something I notice on RTT, about its predictions for delayed trains (in practice this only really has relevance for freight).

Say the schedule has two timing points, A and B, and Z somewhere further down the line, and the timings look like this
A - 1200
B - 1205
...
Z - 1230

The train passes A at 1145, so appears to be running 15 minutes early, and an estimated passing time of 1150 is shown for B and 1215 is shown for Z.

But at 1200 it's still not passed B. You'd think the system would recognise that the earliest it could now possibly pass B is 1201 so it could predict this, and 1226 for Z. Or even assume in the absence of other information that it is just going to run in its pathed slot, and revert the predicted timings to the scheduled ones.

That's not what happens though - it'll carry on showing 1215 for Z even if it's still not shown up at B by 1205 and even if time ticks on past 1215 without it showing up at B!

What's the reasoning for this - is it that it might just not have registered at all timing points and therefore the fact that it's passed A is considered of greater predictive value than the fact that it hasn't registered at B?