• Our booking engine at tickets.railforums.co.uk (powered by TrainSplit) helps support the running of the forum with every ticket purchase! Find out more and ask any questions/give us feedback in this thread!

Network Rail Open Data

Status
Not open for further replies.

BIBBY

Member
Joined
10 Jul 2012
Messages
14
Location
Derby
I'm trying to get a .Net Client to work and having no luck at all ... have tried Telnet, Putty and FireZilla but cannot even connect using these .. any ideas anyone ??
 
Sponsor Post - registered members do not see these adverts; click here to register, or click here to log in
R

RailUK Forums

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
Are you able to telnet to datafeeds.networkrail.co.uk on port 61618? If not, check the machine you're using doesn't have a firewall or antivirus software preventing you from connecting out on this port.

FileZilla won't help much, as it's an FTP client.
 

BIBBY

Member
Joined
10 Jul 2012
Messages
14
Location
Derby
I've tried using telnet from several different locations (at home and at work) I've got the Network guys to test as well (inside and outside of our network) and they can't get a connection. Would love to use the datafeeds if only I could get access !!!
 

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
What happens? Do you get a 'Connection refused' message? Does it time out when connecting?

On my machine here, I get:

piccadilly:~ pwh$ telnet datafeeds.networkrail.co.uk 61618
Trying 46.137.163.200...
Connected to ntrodprodlb1-1615579077.eu-west-1.elb.amazonaws.com (46.137.163.200).
Escape character is '^]'.

If you can get that far, type "CONNECT", then enter twice, then send a ^@ by pressing CTRL + SPACE and pressing enter. That should elicit a response.
 

BIBBY

Member
Joined
10 Jul 2012
Messages
14
Location
Derby
Welcome to Microsoft Telnet Client

Escape Character is 'CTRL+]'
Microsoft Telnet> open datafeeds.networkrail.co.uk 61618
Connecting To datafeeds.networkrail.co.uk...Could not open connection to the hos
t, on port 61618: Connect failed
Microsoft Telnet>
--- old post above --- --- new post below ---
I'm just not able to establish a connection at all :(
 

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
I'll drop you a private message
--- old post above --- --- new post below ---
Welcome to Microsoft Telnet Client

Escape Character is 'CTRL+]'
Microsoft Telnet> open datafeeds.networkrail.co.uk 61618
Connecting To datafeeds.networkrail.co.uk...Could not open connection to the hos
t, on port 61618: Connect failed
Microsoft Telnet>
--- old post above --- --- new post below ---
I'm just not able to establish a connection at all :(

OK, I can't send you a private message. Can you send me a private message with some IM contact details for you?
 

b0b

Established Member
Joined
25 Jan 2010
Messages
1,332
Anyone else still seeing missing berth step data?
 

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
Yup. I'm currently working out how much of it is missing, and whether it's related to time-of-day etc.
 

gavco98uk

Member
Joined
8 Jul 2012
Messages
7
Location
Southwest Scotland
I haven't done much with the TD data yet, as I don't have a list of the berth steps yet, but I have been using the train Movement data, and there seem to be a lot of messages missing from this.

For example there are no messages being reported from Scotland, and messages from the rest of the country seem to be sporadic.
 

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
I haven't done much with the TD data yet, as I don't have a list of the berth steps yet, but I have been using the train Movement data, and there seem to be a lot of messages missing from this.

For example there are no messages being reported from Scotland, and messages from the rest of the country seem to be sporadic.

Are you subscribed to the all-TOC movement feed? How do you identify whether you've lost messages or not? Are you expecting every train to report at every station?
 

gavco98uk

Member
Joined
8 Jul 2012
Messages
7
Location
Southwest Scotland
Are you subscribed to the all-TOC movement feed? How do you identify whether you've lost messages or not? Are you expecting every train to report at every station?

Yes, I am subscribed to the all-TOC movement feed. IMy system maintains a permanent connection to the feed and continually receives updates, so it shouldnt be missing any messages. Yes, i was expecting that a train would report in at every station, or at least at most of them. Is this not the case?
 

The Planner

Veteran Member
Joined
15 Apr 2008
Messages
16,130
Only if it stops there or it is a mandatory timing point, if it is neither then it won't report.
 

Paul Kelly

Verified Rep - BR Fares
Joined
16 Apr 2010
Messages
4,135
Location
Reading
Well you should get a report for all the timing points and station stops shown in the schedule for that train, I would expect. If not then something is wrong. Have you correlated the reported movements against the expected reporting points in the relevant schedule? (You can check this somewhere like www.opentraintimes.com if you haven't developed your own schedule data parser yet.)
 

michael769

Established Member
Joined
9 Oct 2005
Messages
2,006
There are some stations that never get a report at all. Looking at the NRE live departure boards they always show "no report" there too which suggests that some stations simply lack a working monitoring point.

A good example is the Glasgow to Edinburgh via Shotts line which has several stations for which this is the case.
 

swt_passenger

Veteran Member
Joined
7 Apr 2010
Messages
31,559

gavco98uk

Member
Joined
8 Jul 2012
Messages
7
Location
Southwest Scotland
Well you should get a report for all the timing points and station stops shown in the schedule for that train, I would expect. If not then something is wrong. Have you correlated the reported movements against the expected reporting points in the relevant schedule? (You can check this somewhere like www.opentraintimes.com if you haven't developed your own schedule data parser yet.)

To be honest i need to look into it a little more to see what is being reported and what isn't. I wrote a basic program to receive the feeds and log them to file, mainly so i could get a better idea of what data was coming back and the format of this.

From scanning the text file, I didnt see a single result for anywhere in Scotland, no reports from Edinburgh, Glasgow, Aberdeen etc, or even Carlisle. Oxenholme however did appear - nothing north of this though.

I need to have a look at it in a bit more detail, but my initial investigation suspects not all stations are being reported on the feed.

It would be great if someone else could subscribe to the ALL data feed and look out for some reports from trains in Scotland, this will then confirm if it is my dodgy code at fault.
 

b0b

Established Member
Joined
25 Jan 2010
Messages
1,332
From scanning the text file, I didnt see a single result for anywhere in Scotland, no reports from Edinburgh, Glasgow, Aberdeen etc, or even Carlisle. Oxenholme however did appear - nothing north of this though.

Then you must have bugs, I know I've been able to follow EUS-GLC trains "all the way" to Glasgow Central, I'm sure I've seen TRAIN_MVT data reports for Carlisle, Beattock Summit, Lanark Junction, Uddingston Junction, Rutherglen East Junction and arrival at Glasgow Central platform 1.

However I've not seen data for those trains passing say Cambuslang or other intermediate stations, I assume because they're not timing points. Therefore I was trying to map out the TD data so I could interpolate from the berth steps but that TD data is sadly incomplete as of now.
--- old post above --- --- new post below ---
It would be great if someone else could subscribe to the ALL data feed and look out for some reports from trains in Scotland, this will then confirm if it is my dodgy code at fault.

one from my log, I have 1303 just for 07257 (Glasgow Central)
Code:
publishing to TRAIN_MVT_ALL_TOC
{
  "body": {
    "actual_timestamp": "1342283100000",
    "auto_expected": "true",
    "correction_ind": "false",
    "current_train_id": "",
    "delay_monitoring_point": "false",
    "direction_ind": "DOWN",
    "division_code": "60",
    "event_source": "AUTOMATIC",
    "event_type": "DEPARTURE",
    "gbtt_timestamp": "",
    "line_ind": "",
    "loc_stanox": "07246",
    "next_report_run_time": "2",
    "next_report_stanox": "07257",
    "offroute_ind": "false",
    "original_loc_stanox": "",
    "original_loc_timestamp": "",
    "planned_event_type": "DEPARTURE",
    "planned_timestamp": "1342283160000",
    "platform": "",
    "reporting_stanox": "00000",
    "route": "2",
    "timetable_variation": "1",
    "toc_id": "60",
    "train_file_address": null,
    "train_id": "072G60MR14",
    "train_service_code": "13569815",
    "train_terminated": "false",
    "variation_status": "EARLY"
  },
  "header": {
    "msg_queue_timestamp": "1342279529000",
    "msg_type": "0003",
    "original_data_source": "SMART",
    "source_dev_id": "",
    "source_system_id": "TRUST",
    "user_id": ""
  }
}
 

Poggs

Member
Joined
28 Aug 2008
Messages
283
Location
London
Then you must have bugs, I know I've been able to follow EUS-GLC trains "all the way" to Glasgow Central, I'm sure I've seen TRAIN_MVT data reports for Carlisle, Beattock Summit, Lanark Junction, Uddingston Junction, Rutherglen East Junction and arrival at Glasgow Central platform 1.

However I've not seen data for those trains passing say Cambuslang or other intermediate stations, I assume because they're not timing points. Therefore I was trying to map out the TD data so I could interpolate from the berth steps but that TD data is sadly incomplete as of now.

Not so fast - saying somebody has a 'bug' because they might understand isn't good.

A train movement will only trigger a TRUST report when it passes a location in the schedule that is a reporting point, and only automatically if it's an automatic reporting point.

How may missing locations do you have, and what do you think they are?

Incidentally, there is a data set somewhere with all berth steps and the STANOXes, lines, directions and event types that SMART uses to produce TRUST reports.
 

b0b

Established Member
Joined
25 Jan 2010
Messages
1,332
Not so fast - saying somebody has a 'bug' because they might understand isn't good.

I assumed he had a bug if he said he's subscribing to TRAIN_MVT_ALL_TOC and doesnt see reports for any STANOX's in Scotland.


Incidentally, there is a data set somewhere with all berth steps and the STANOXes, lines, directions and event types that SMART uses to produce TRUST reports.

please define 'somewhere' :D
 

gavco98uk

Member
Joined
8 Jul 2012
Messages
7
Location
Southwest Scotland
oh fiddlesticks! (well, a word similar to that anyway!)

Based on your example of Glasgow Central, I just did a quick search of my DB to make sure I had the right code for Glasgow Central, and got no result for 07257. I thought that was odd, so took a look in the database to make sure they were all there... searching for Aberdeen I found "2057"... yea, when I imported the values it has kindly chopped the 0 off the start of the code!

Let me guess, every code in Scotland starts with a "0"? Might also explain why other trains also seemed to report in rather sporadically!

Again, fiddlesticks!
 

Amy

Member
Joined
19 Jul 2012
Messages
10
I've been trying to get the data feeds for three days now...haven't been successful :s

This is my php client

<?php
require_once("Stomp.php");
// Network Rail Stomp Handler example by ian13
$server = "tcp://datafeeds.networkrail.co.uk:61618";
$user = "loginUsername";
$password = "loginPassword";
$channel = "TRAIN_MVT_ALL_TOC";

$con = new Stomp($server, $user, $password);
if (!$con) {
die('Connection failed: ' . stomp_connect_error());
}
$con->subscribe("/topic/" . $channel);
while($con){
if ($con->hasFrame()){
$msg = $con->readFrame();
foreach (json_decode($msg->body) as $event) {
// do stuff with $event here
}
$con->ack($msg);
}
}
die('Connection lost: ' . time());
?>

I get the following errors:

Uncaught exception 'StompException' with message 'Socket connection hasn't been established'

Or

User name [ ] or password is invalid

Please note that I have installed activemq and Stomp and was able to run the first.php example successfully.

Any help will be very much appreciated

Regards,
Amy
 

gavco98uk

Member
Joined
8 Jul 2012
Messages
7
Location
Southwest Scotland
Anyone used a C# .Net Client to connect ? .. I'm still struggling

First you need to download ApacheNMS.Stomp Client from https://cwiki.apache.org/NMS/apachenmsstomp-v152.html. Add this as a reference to your project.

After that, the following code will get you started:
Code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using Apache.NMS;

namespace ApacheNMS
{
    class Program
    {
        static void Main(string[] args)
        {
            IConnectionFactory factory = new NMSConnectionFactory(new Uri("stomp:tcp://datafeed:61618"));

            IConnection connection = factory.CreateConnection("username", "password");
            ISession session = connection.CreateSession();

            IDestination destination = session.GetDestination("topic://" + "DATA_FEED");
            IMessageConsumer consumer = session.CreateConsumer(destination);


            connection.Start();
            consumer.Listener += new MessageListener(OnMessage);
            Console.WriteLine("Consumer started, waiting for messages... (Press ENTER to stop.)");

            Console.ReadLine();
            connection.Close();
        }

        private static void OnMessage(IMessage message)
        {
            try
            {
                Console.WriteLine("Median-Server (.NET): Message received");

                ITextMessage msg = (ITextMessage)message;
                message.Acknowledge();

                Console.WriteLine(msg.Text);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                Console.WriteLine("---");
                Console.WriteLine(ex.InnerException);
                Console.WriteLine("---");
                Console.WriteLine(ex.InnerException.Message);
            }
        }
    }
}

For more details on what to do, I have written a post about this on my blog: http://gavincoates.com/c-activemq-stomp
 

BIBBY

Member
Joined
10 Jul 2012
Messages
14
Location
Derby
Thanks for your help with this code but I'm still getting

The thread '<No Name>' (0x22cc) has exited with code 0 (0x0).
The thread '<No Name>' (0x23f8) has exited with code 0 (0x0).
The thread '<No Name>' (0x219c) has exited with code 0 (0x0).
A first chance exception of type 'System.IO.IOException' occurred in System.dll
The thread '<No Name>' (0x2280) has exited with code 0 (0x0).
Consumer started, waiting for messages... (Press ENTER to stop.)
The thread '<No Name>' (0x1650) has exited with code 0 (0x0).
 

b0b

Established Member
Joined
25 Jan 2010
Messages
1,332
try replacing datafeed in the following line:

IConnectionFactory factory = new NMSConnectionFactory(new Uri("stomp:tcp://datafeed:61618"));


with datafeeds.networkrail.co.uk
 

BIBBY

Member
Joined
10 Jul 2012
Messages
14
Location
Derby
Got it working .. it was the topic I hadn't replaced
Thanks so much for your help !!!
 

Amy

Member
Joined
19 Jul 2012
Messages
10
How can we procees the JSON message via php. Are there any ID - Value maps?

Hello Poggs Can I please have the list of IDs
--- old post above --- --- new post below ---
I've been trying to get the data feeds for three days now...haven't been successful :s

This is my php client

<?php
require_once("Stomp.php");
// Network Rail Stomp Handler example by ian13
$server = "tcp://datafeeds.networkrail.co.uk:61618";
$user = "loginUsername";
$password = "loginPassword";
$channel = "TRAIN_MVT_ALL_TOC";

$con = new Stomp($server, $user, $password);
if (!$con) {
die('Connection failed: ' . stomp_connect_error());
}
$con->subscribe("/topic/" . $channel);
while($con){
if ($con->hasFrame()){
$msg = $con->readFrame();
foreach (json_decode($msg->body) as $event) {
// do stuff with $event here
}
$con->ack($msg);
}
}
die('Connection lost: ' . time());
?>

I get the following errors:

Uncaught exception 'StompException' with message 'Socket connection hasn't been established'

Or

User name [ ] or password is invalid

Please note that I have installed activemq and Stomp and was able to run the first.php example successfully.

Any help will be very much appreciated

Regards,
Amy

It was my stomp library.

I get the JSON reply now but need to interpret the IDs. Anyone has city ID-Value table....etc
 

ralphchadkirk

Established Member
Joined
20 Oct 2008
Messages
5,753
Location
Essex
Stomp will generate an associative array or an object depending on choice from the message.


Sent from my iPhone 4 using Tapatalk
 
Status
Not open for further replies.

Top