Network Rail Open Data

[BIBBY]

I'm trying to get a .Net Client to work and having no luck at all ... have tried Telnet, Putty and FireZilla but cannot even connect using these .. any ideas anyone ??

 

[Poggs]

Are you able to telnet to datafeeds.networkrail.co.uk on port 61618? If not, check the machine you're using doesn't have a firewall or antivirus software preventing you from connecting out on this port.

FileZilla won't help much, as it's an FTP client.

 

[BIBBY]

I've tried using telnet from several different locations (at home and at work) I've got the Network guys to test as well (inside and outside of our network) and they can't get a connection. Would love to use the datafeeds if only I could get access !!!

 

[Poggs]

What happens? Do you get a 'Connection refused' message? Does it time out when connecting?

On my machine here, I get:

piccadilly:~ pwh$ telnet datafeeds.networkrail.co.uk 61618
Trying 46.137.163.200...
Connected to ntrodprodlb1-1615579077.eu-west-1.elb.amazonaws.com (46.137.163.200).
Escape character is '^]'.

If you can get that far, type "CONNECT", then enter twice, then send a ^@ by pressing CTRL + SPACE and pressing enter. That should elicit a response.

 

[BIBBY]

Welcome to Microsoft Telnet Client

Escape Character is 'CTRL+]'
Microsoft Telnet> open datafeeds.networkrail.co.uk 61618
Connecting To datafeeds.networkrail.co.uk...Could not open connection to the hos
t, on port 61618: Connect failed
Microsoft Telnet>
--- old post above --- --- new post below ---
I'm just not able to establish a connection at all

 

[Poggs]

I'll drop you a private message
--- old post above --- --- new post below ---

Welcome to Microsoft Telnet Client

Escape Character is 'CTRL+]'
Microsoft Telnet> open datafeeds.networkrail.co.uk 61618
Connecting To datafeeds.networkrail.co.uk...Could not open connection to the hos
t, on port 61618: Connect failed
Microsoft Telnet>
--- old post above --- --- new post below ---
I'm just not able to establish a connection at all

OK, I can't send you a private message. Can you send me a private message with some IM contact details for you?

 

[BIBBY]

I can't IM at work but here's my email <email address removed - Mike395>

 

[Poggs]

I can't IM at work but here's my email <email address removed - Mike395>

For the benefit of everyone else, it turned out to be a firewall problem.

 

[b0b]

Anyone else still seeing missing berth step data?

 

[Poggs]

Yup. I'm currently working out how much of it is missing, and whether it's related to time-of-day etc.

 

[gavco98uk]

I haven't done much with the TD data yet, as I don't have a list of the berth steps yet, but I have been using the train Movement data, and there seem to be a lot of messages missing from this.

For example there are no messages being reported from Scotland, and messages from the rest of the country seem to be sporadic.

 

[Poggs]

I haven't done much with the TD data yet, as I don't have a list of the berth steps yet, but I have been using the train Movement data, and there seem to be a lot of messages missing from this.

For example there are no messages being reported from Scotland, and messages from the rest of the country seem to be sporadic.

Are you subscribed to the all-TOC movement feed? How do you identify whether you've lost messages or not? Are you expecting every train to report at every station?

 

[gavco98uk]

Are you subscribed to the all-TOC movement feed? How do you identify whether you've lost messages or not? Are you expecting every train to report at every station?

Yes, I am subscribed to the all-TOC movement feed. IMy system maintains a permanent connection to the feed and continually receives updates, so it shouldnt be missing any messages. Yes, i was expecting that a train would report in at every station, or at least at most of them. Is this not the case?

 

[The Planner]

Only if it stops there or it is a mandatory timing point, if it is neither then it won't report.

 

[Paul Kelly]

Well you should get a report for all the timing points and station stops shown in the schedule for that train, I would expect. If not then something is wrong. Have you correlated the reported movements against the expected reporting points in the relevant schedule? (You can check this somewhere like www.opentraintimes.com if you haven't developed your own schedule data parser yet.)

 

[michael769]

There are some stations that never get a report at all. Looking at the NRE live departure boards they always show "no report" there too which suggests that some stations simply lack a working monitoring point.

A good example is the Glasgow to Edinburgh via Shotts line which has several stations for which this is the case.

 

[swt_passenger]

There are some stations that never get a report at all. Looking at the NRE live departure boards they always show "no report" there too which suggests that some stations simply lack a working monitoring point.

They do attempt to explain that, without saying specifically that some stations will never report anything:

http://www.nationalrail.co.uk/times_fares/live_departure_boards/limitations.html

 

[gavco98uk]

Well you should get a report for all the timing points and station stops shown in the schedule for that train, I would expect. If not then something is wrong. Have you correlated the reported movements against the expected reporting points in the relevant schedule? (You can check this somewhere like www.opentraintimes.com if you haven't developed your own schedule data parser yet.)

To be honest i need to look into it a little more to see what is being reported and what isn't. I wrote a basic program to receive the feeds and log them to file, mainly so i could get a better idea of what data was coming back and the format of this.

From scanning the text file, I didnt see a single result for anywhere in Scotland, no reports from Edinburgh, Glasgow, Aberdeen etc, or even Carlisle. Oxenholme however did appear - nothing north of this though.

I need to have a look at it in a bit more detail, but my initial investigation suspects not all stations are being reported on the feed.

It would be great if someone else could subscribe to the ALL data feed and look out for some reports from trains in Scotland, this will then confirm if it is my dodgy code at fault.

 

[b0b]

From scanning the text file, I didnt see a single result for anywhere in Scotland, no reports from Edinburgh, Glasgow, Aberdeen etc, or even Carlisle. Oxenholme however did appear - nothing north of this though.

Then you must have bugs, I know I've been able to follow EUS-GLC trains "all the way" to Glasgow Central, I'm sure I've seen TRAIN_MVT data reports for Carlisle, Beattock Summit, Lanark Junction, Uddingston Junction, Rutherglen East Junction and arrival at Glasgow Central platform 1.

However I've not seen data for those trains passing say Cambuslang or other intermediate stations, I assume because they're not timing points. Therefore I was trying to map out the TD data so I could interpolate from the berth steps but that TD data is sadly incomplete as of now.
--- old post above --- --- new post below ---

It would be great if someone else could subscribe to the ALL data feed and look out for some reports from trains in Scotland, this will then confirm if it is my dodgy code at fault.

one from my log, I have 1303 just for 07257 (Glasgow Central)

publishing to TRAIN_MVT_ALL_TOC
{
  "body": {
    "actual_timestamp": "1342283100000",
    "auto_expected": "true",
    "correction_ind": "false",
    "current_train_id": "",
    "delay_monitoring_point": "false",
    "direction_ind": "DOWN",
    "division_code": "60",
    "event_source": "AUTOMATIC",
    "event_type": "DEPARTURE",
    "gbtt_timestamp": "",
    "line_ind": "",
    "loc_stanox": "07246",
    "next_report_run_time": "2",
    "next_report_stanox": "07257",
    "offroute_ind": "false",
    "original_loc_stanox": "",
    "original_loc_timestamp": "",
    "planned_event_type": "DEPARTURE",
    "planned_timestamp": "1342283160000",
    "platform": "",
    "reporting_stanox": "00000",
    "route": "2",
    "timetable_variation": "1",
    "toc_id": "60",
    "train_file_address": null,
    "train_id": "072G60MR14",
    "train_service_code": "13569815",
    "train_terminated": "false",
    "variation_status": "EARLY"
  },
  "header": {
    "msg_queue_timestamp": "1342279529000",
    "msg_type": "0003",
    "original_data_source": "SMART",
    "source_dev_id": "",
    "source_system_id": "TRUST",
    "user_id": ""
  }
}

 

[Poggs]

Then you must have bugs, I know I've been able to follow EUS-GLC trains "all the way" to Glasgow Central, I'm sure I've seen TRAIN_MVT data reports for Carlisle, Beattock Summit, Lanark Junction, Uddingston Junction, Rutherglen East Junction and arrival at Glasgow Central platform 1.

However I've not seen data for those trains passing say Cambuslang or other intermediate stations, I assume because they're not timing points. Therefore I was trying to map out the TD data so I could interpolate from the berth steps but that TD data is sadly incomplete as of now.

Not so fast - saying somebody has a 'bug' because they might understand isn't good.

A train movement will only trigger a TRUST report when it passes a location in the schedule that is a reporting point, and only automatically if it's an automatic reporting point.

How may missing locations do you have, and what do you think they are?

Incidentally, there is a data set somewhere with all berth steps and the STANOXes, lines, directions and event types that SMART uses to produce TRUST reports.

 

[b0b]

Not so fast - saying somebody has a 'bug' because they might understand isn't good.

I assumed he had a bug if he said he's subscribing to TRAIN_MVT_ALL_TOC and doesnt see reports for any STANOX's in Scotland.

Incidentally, there is a data set somewhere with all berth steps and the STANOXes, lines, directions and event types that SMART uses to produce TRUST reports.

please define 'somewhere'

 

[gavco98uk]

oh fiddlesticks! (well, a word similar to that anyway!)

Based on your example of Glasgow Central, I just did a quick search of my DB to make sure I had the right code for Glasgow Central, and got no result for 07257. I thought that was odd, so took a look in the database to make sure they were all there... searching for Aberdeen I found "2057"... yea, when I imported the values it has kindly chopped the 0 off the start of the code!

Let me guess, every code in Scotland starts with a "0"? Might also explain why other trains also seemed to report in rather sporadically!

Again, fiddlesticks!

 

[BIBBY]

Anyone used a C# .Net Client to connect ? .. I'm still struggling

 

[Amy]

I've been trying to get the data feeds for three days now...haven't been successful

This is my php client

<?php
require_once("Stomp.php");
// Network Rail Stomp Handler example by ian13
$server = "tcp://datafeeds.networkrail.co.uk:61618";
$user = "loginUsername";
$password = "loginPassword";
$channel = "TRAIN_MVT_ALL_TOC";

$con = new Stomp($server, $user, $password);
if (!$con) {
die('Connection failed: ' . stomp_connect_error());
}
$con->subscribe("/topic/" . $channel);
while($con){
if ($con->hasFrame()){
$msg = $con->readFrame();
foreach (json_decode($msg->body) as $event) {
// do stuff with $event here
}
$con->ack($msg);
}
}
die('Connection lost: ' . time());
?>​

I get the following errors:

Uncaught exception 'StompException' with message 'Socket connection hasn't been established'

Or

User name [ ] or password is invalid

Please note that I have installed activemq and Stomp and was able to run the first.php example successfully.

Any help will be very much appreciated

Regards,
Amy

 

[gavco98uk]

Anyone used a C# .Net Client to connect ? .. I'm still struggling

First you need to download ApacheNMS.Stomp Client from https://cwiki.apache.org/NMS/apachenmsstomp-v152.html. Add this as a reference to your project.

After that, the following code will get you started:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using Apache.NMS;

namespace ApacheNMS
{
    class Program
    {
        static void Main(string[] args)
        {
            IConnectionFactory factory = new NMSConnectionFactory(new Uri("stomp:tcp://datafeed:61618"));

            IConnection connection = factory.CreateConnection("username", "password");
            ISession session = connection.CreateSession();

            IDestination destination = session.GetDestination("topic://" + "DATA_FEED");
            IMessageConsumer consumer = session.CreateConsumer(destination);


            connection.Start();
            consumer.Listener += new MessageListener(OnMessage);
            Console.WriteLine("Consumer started, waiting for messages... (Press ENTER to stop.)");

            Console.ReadLine();
            connection.Close();
        }

        private static void OnMessage(IMessage message)
        {
            try
            {
                Console.WriteLine("Median-Server (.NET): Message received");

                ITextMessage msg = (ITextMessage)message;
                message.Acknowledge();

                Console.WriteLine(msg.Text);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                Console.WriteLine("---");
                Console.WriteLine(ex.InnerException);
                Console.WriteLine("---");
                Console.WriteLine(ex.InnerException.Message);
            }
        }
    }
}

For more details on what to do, I have written a post about this on my blog: http://gavincoates.com/c-activemq-stomp

 

[BIBBY]

Thanks for your help with this code but I'm still getting

The thread '<No Name>' (0x22cc) has exited with code 0 (0x0).
The thread '<No Name>' (0x23f8) has exited with code 0 (0x0).
The thread '<No Name>' (0x219c) has exited with code 0 (0x0).
A first chance exception of type 'System.IO.IOException' occurred in System.dll
The thread '<No Name>' (0x2280) has exited with code 0 (0x0).
Consumer started, waiting for messages... (Press ENTER to stop.)
The thread '<No Name>' (0x1650) has exited with code 0 (0x0).

 

[b0b]

try replacing datafeed in the following line:

IConnectionFactory factory = new NMSConnectionFactory(new Uri("stomp:tcp://datafeed:61618"));


with datafeeds.networkrail.co.uk

 

[BIBBY]

Got it working .. it was the topic I hadn't replaced
Thanks so much for your help !!!

 

[Amy]

How can we procees the JSON message via php. Are there any ID - Value maps?

Hello Poggs Can I please have the list of IDs
--- old post above --- --- new post below ---

I've been trying to get the data feeds for three days now...haven't been successful

This is my php client

<?php
require_once("Stomp.php");
// Network Rail Stomp Handler example by ian13
$server = "tcp://datafeeds.networkrail.co.uk:61618";
$user = "loginUsername";
$password = "loginPassword";
$channel = "TRAIN_MVT_ALL_TOC";

$con = new Stomp($server, $user, $password);
if (!$con) {
die('Connection failed: ' . stomp_connect_error());
}
$con->subscribe("/topic/" . $channel);
while($con){
if ($con->hasFrame()){
$msg = $con->readFrame();
foreach (json_decode($msg->body) as $event) {
// do stuff with $event here
}
$con->ack($msg);
}
}
die('Connection lost: ' . time());
?>​

I get the following errors:

Uncaught exception 'StompException' with message 'Socket connection hasn't been established'

Or

User name [ ] or password is invalid

Please note that I have installed activemq and Stomp and was able to run the first.php example successfully.

Any help will be very much appreciated

Regards,
Amy

It was my stomp library.

I get the JSON reply now but need to interpret the IDs. Anyone has city ID-Value table....etc

 

[ralphchadkirk]

Stomp will generate an associative array or an object depending on choice from the message.


Sent from my iPhone 4 using Tapatalk