• Our booking engine at tickets.railforums.co.uk (powered by TrainSplit) helps support the running of the forum with every ticket purchase! Find out more and ask any questions/give us feedback in this thread!

Network Rail Open Data

Status
Not open for further replies.

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
Out of interest, you say 'I'. Does that mean you had some influence in the choice of platform? (And no problem about the neutral standard. I totally understand that you need something that can be accessed from any platform).

I've had access to the feed from TD.net for the past year, and my goal has always to be to lower the barrier of entry to anyone who wants to consume the data. Stomp was the natural choice - had I wanted a full feed directly from Network Rail, it's an IBM WebSphere MQ server and several thousand pounds! I fed this back up the food chain.

On the schedules, do you need to have a your own Amazon S3 account set up to download them? I don't and I had assumed that was fine since the URL to download schedules is a networkrail one, not an amazon one, but now wondering if that's the problem. I see one of my download attempts has caused a page redirect to https://nr-datafeed-cif.s3.amazonaws.com/.

You don't need an Amazon account to download the schedules - S3 is the content delivery network. You will, however, need to call https://datafeeds.networkrail.co.uk/ntrod /CifFileAuthenticate?type=CIF_ALL_UPDATE_DAILY&day=tocupdate-
mon and authenticate with your username/password to be redirected to a temporary URL where you can download the schedule files.
 
Sponsor Post - registered members do not see these adverts; click here to register, or click here to log in
R

RailUK Forums

Zoe

Established Member
Joined
22 Aug 2008
Messages
5,905
I have been watching the TD feed for one area in the last few days and I have noticed that at times, I don't get a message for each berth step. For example I may get some berth step messages for a specific train but then no messages of any berth steps for that train for a time and then when I do get a mesasage, I notice that some berth steps have not been reported. When it's very quiet there doesn't seem to be any issue though as earlier I observed 1A40 and 1C99 through the Bristol area and every berth step was reported.
 
Last edited:

ralphchadkirk

Established Member
Joined
20 Oct 2008
Messages
5,753
Location
Essex
I have been watching the TD feed for one area in the last few days and I have noticed that at times, I don't get a message for each berth step. For example I may get some berth step messages for a specific train but then no messages of any berth steps for that train for a time and then when I do get a mesasage, I notice that some berth steps have not been reported. When it's very quiet there doesn't seem to be any issue though as earlier I observed 1A40 and 1C99 through the Bristol area and every berth step was reported.

Manual input area?
 

Zoe

Established Member
Joined
22 Aug 2008
Messages
5,905
Manual input area?
No, these are PSB areas. It's also not specific berths, it's different with each train, sometimes I can be very lucky like earlier when I every berth step on 1A40 and 1C99 was reported but other times quite a few berth steps are not getting reported although exactly which ones seems to change with each train.
 
Last edited:

Paul Kelly

Verified Rep - BR Fares
Joined
16 Apr 2010
Messages
4,135
Location
Reading
I've been following the train movements feed and noticed there was an outage this morning, actually one shortage from 09:36-09:37 and then a much longer one from 09:40-10:38, where there were way fewer reports than normal coming through. Since 10:39 it's been back to normal though.

I saw the same behaviour yesterday afternoon for a while too, but don't have the exact times. Just wondering if those times this morning correspond with when anybody else is seeing strangeness?
 

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
I've been following the train movements feed and noticed there was an outage this morning, actually one shortage from 09:36-09:37 and then a much longer one from 09:40-10:38, where there were way fewer reports than normal coming through. Since 10:39 it's been back to normal though.

I saw the same behaviour yesterday afternoon for a while too, but don't have the exact times. Just wondering if those times this morning correspond with when anybody else is seeing strangeness?

I've noticed this too - let me get some feedback sent... several of the things I'm working on simply won't work very will with missing berth steps!
 

Zoe

Established Member
Joined
22 Aug 2008
Messages
5,905
I've been following the train movements feed and noticed there was an outage this morning, actually one shortage from 09:36-09:37 and then a much longer one from 09:40-10:38, where there were way fewer reports than normal coming through. Since 10:39 it's been back to normal though.

I saw the same behaviour yesterday afternoon for a while too, but don't have the exact times. Just wondering if those times this morning correspond with when anybody else is seeing strangeness?
Yes, I noticed that. Strangely after things went back to normal I had about 30 minutes or so of getting a lot more TD berth step reports before it went back to the current situation where quite a few do not seem to be reported.
 
Last edited:

DarloRich

Veteran Member
Joined
12 Oct 2010
Messages
29,396
Location
Fenny Stratford
I am sorry are you people even speaking English???????????

I think this is a very bad idea. I mean if the spotters have this data how will i look connected and important? Mere normals shouldnt know how things work! This is the thin end of the wedge! Next you will work out that most of it is just pushing buttons, pulling levers, drinking tea and reading the paper FFS!

Where is my union? :lol::lol::lol::D
 

Aictos

Established Member
Joined
28 Apr 2009
Messages
10,403
Ooh ooh ooh! I would say centred around Stevenage or around Finsbury Park.. well I would wouldn't I? ;)

Hmm, I would be interested in either Edinburgh/Newcastle to Doncaster and Leeds to Doncaster :)
 

b0b

Established Member
Joined
25 Jan 2010
Messages
1,332
Can someone explain what the "berth" data is?
 

ralphchadkirk

Established Member
Joined
20 Oct 2008
Messages
5,753
Location
Essex
Can someone explain what the "berth" data is?

They are train description berths; basically a small dot matrix screen by signals (not all though) on the signaller's panel (or computer if a lever frame, or even just as part of the display at IECCs) which 'describes' the train - it shows the head code. These can either be interposed, where the signaller changes or enters a description, cancelled, when a description is removed, or stepped when the train moves to the next signal section so its associated description moves to the next berth along.
 

b0b

Established Member
Joined
25 Jan 2010
Messages
1,332
These can either be interposed, where the signaller changes or enters a description, cancelled, when a description is removed, or stepped when the train moves to the next signal section so its associated description moves to the next berth along.

so can i assume its a bit like a signal block section, or does that simplify it too much?
 

ralphchadkirk

Established Member
Joined
20 Oct 2008
Messages
5,753
Location
Essex
so can i assume its a bit like a signal block section, or does that simplify it too much?

A simplification would be that a berth is a screen that tells the signaller what is in each signal section.
 

b0b

Established Member
Joined
25 Jan 2010
Messages
1,332
so how do I get a map of where the berths are located?
 

DynamicSpirit

Established Member
Joined
12 Apr 2012
Messages
8,268
Location
SE London
DynamicSpirit said:
Originally Posted by DynamicSpirit
Has anyone had any luck downloading the train schedules? I don't seem to be able to.

I have just tried: https://datafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ALL_FULL_DAILY&day=toc-full

It works for me, I am subscribed to the schedule "All Full Daily"

On more investigation it seems that the one you tried - the ALL_FULL one - is working, but is almost the only file that is.

The ALL_UPDATE files for Thursday-Sunday are also there, but Mon-Wed aren't. I'm guessing that may be an artifact of this being the very first week of data, and the remaining ones will be there by Friday.

However, none of the files for individual TOCs are working - whenever I try any of them I get the 401 not subscribed error. On more experimenting, this error happens either if you download something that you're not subscribed to, or if you mis-type something and so try to download a file that doesn't exist. So I would hypothesize that either there's a bug on the system - it's not recognizing individual TOC subscriptions, or the sample URL in the developer toolkit (https://datafeeds.networkrail.co.uk/ntrod/CifFileAuthenticate?type=CIF_ED_FULL_DAILY&day=toc-full) is wrong
 

ian13

Member
Joined
17 Jul 2008
Messages
166
However, none of the files for individual TOCs are working - whenever I try any of them I get the 401 not subscribed error. On more experimenting, this error happens either if you download something that

They're not currently available - they will be, but there are other issues with the schedule data that are being worked on first.
 

DAH

Member
Joined
17 Apr 2008
Messages
19
Having the same problem, just will not connect even though my status is 'Active'.

Getting:

Code:
Fatal error: Uncaught exception 'StompException' with message 'Could not connect to datafeeds.networkrail.co.uk:61618 (10/10)' in /<path>/Stomp.php:168
Stack trace: 
#0 <path>/Stomp.php(190): Stomp->_makeConnection() 
#1 <path>/stomp_test.php(7): Stomp->connect() 
#2 {main} thrown in <path>/Stomp.php on line 168

stomp_test.php is using what is suggested in the Developer Pack V4.0.

OK,

Out of curiosity, I just tried running this from the localhost on my PC and got instant results on train movements. Uploaded the exact same files to my web server and received the same error messages again(unable to connect etc).

Is there a logical reason for this?
 

dosxuk

Established Member
Joined
2 Jan 2011
Messages
1,793
OK,

Out of curiosity, I just tried running this from the localhost on my PC and got instant results on train movements. Uploaded the exact same files to my web server and received the same error messages again(unable to connect etc).

Is there a logical reason for this?

The port could be firewalled on the web server, blocking the connection attempt before it even leaves the machine.
 

DAH

Member
Joined
17 Apr 2008
Messages
19
The port could be firewalled on the web server, blocking the connection attempt before it even leaves the machine.

Cheers for that suggestion, 5 minutes live chat and the port has been openned.

Thank you
 

headshot119

Established Member
Joined
31 Dec 2010
Messages
2,051
Location
Dubai
I've set myself up a program in Ruby just to download the raw TD step data. I'm having trouble accessing anything other than the M&C East Midlands feed. I can't seem to find a way to work out the URLs for all the different feeds.

For example I know that M&C East Midlands is /topic/TD_MC_EM_SIG_AREA.

But what do I use for Western Wales & Marches?


Would have helped if I'd have read the developer pack all the way from top to bottom.

Has anyone got a list, or knows where I can find a list to interpret the area IDs?
 
Last edited:

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
I've set myself up a program in Ruby just to download the raw TD step data. I'm having trouble accessing anything other than the M&C East Midlands feed. I can't seem to find a way to work out the URLs for all the different feeds.

For example I know that M&C East Midlands is /topic/TD_MC_EM_SIG_AREA.

But what do I use for Western Wales & Marches?


Would have helped if I'd have read the developer pack all the way from top to bottom.

Has anyone got a list, or knows where I can find a list to interpret the area IDs?

I have a list here - drop me a private message with your email address and I'll forward 'em on.
 

robh

Member
Joined
30 Jun 2012
Messages
7
Anyone developing a Java client I've come across ActiveMQ Apollo which is a Stomp message broker but the binary packages on the website contains some useful examples. If you extract the archive and browse to examples/stomp/java you can see an implementation of a stomp client that uses the fusesource stompjms library.

Here is a modified version of the Listener example to work with the network rail datafeeds. You will probably want to make use of a JSON library to get anything useful out of the messages but this shows the basic idea:

Code:
import org.fusesource.stomp.jms.*;
import javax.jms.*;

class Listener {

    public static void main(String[] args) throws JMSException {

        String user = "your_username";
        String password = "your_password";
        String host = "datafeeds.networkrail.co.uk";
        int port = 61618;
        String destination = "/topic/TRAIN_MVT_ALL_TOC"; // A topic to which you are subscribed

        StompJmsConnectionFactory factory = new StompJmsConnectionFactory();
        factory.setBrokerURI("tcp://" + host + ":" + port);

        Connection connection = factory.createConnection(user, password);
        connection.start();
        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
        Destination dest = new StompJmsDestination(destination);

        MessageConsumer consumer = session.createConsumer(dest);
        
        System.out.println("Waiting for messages...");
        
        while(true) {
            Message msg = consumer.receive();
            // Process message here
        }

    }
}
 

DAH

Member
Joined
17 Apr 2008
Messages
19
Would anyone know where i could find a reference file which will help with TOC_ID and cancellation codes within the train movement feed??

ie:
[toc_id] => 71
[canx_reason_code] => X2

Many thanks
 

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
Would anyone know where i could find a reference file which will help with TOC_ID and cancellation codes within the train movement feed??

ie:
[toc_id] => 71
[canx_reason_code] => X2

Many thanks

I don't have a list of TOC IDs, but the train_id of the previously activated train that's been cancelled will have the schedule UID in it, and from there you can find out the TOC.

The cancellation reason codes are as per the Delay Attribution Guide at http://www.delayattributionboard.co.uk/documents/dag_pdac/APRIL 1ST 2011 DELAY ATTRIBUTION GUIDE.pdf.
 
Status
Not open for further replies.

Top