Network Rail Open Data

[Zoe]

There seem to be some server issues today, I keep getting disconnected and sometimes can't even connect at all.

 

[Poggs]

There seem to be some server issues today, I keep getting disconnected and sometimes can't even connect at all.

A workaround I've found is to use a durable topic subscriber - you need to set a 'client-id' when you connect and set 'activemq.subscriptionName' to something sensible, e.g. the email address you use, in the SUBSCRIBE frame.

If/when you disconnect, several minutes of messages will queue up and be delivered to you when you reconnect.

 

[Zoe]

The overnight blackout seems to have been extended to now cover the period from 1 AM to 4:09 AM. Additionaly I have noticed that some messages are missing between 11 PM and 1 AM although nowhere near on the scale of the blackout.

 

[AD1]

Hi I'm receiving an error on the line IMessageConsumer with

The connection is already closed!
any ideas?

IDestination destination = session.GetDestination("topic://" + "DATA_FEED");
IMessageConsumer consumer = session.CreateConsumer(destination);

 

[Poggs]

Hi I'm receiving an error on the line IMessageConsumer with

The connection is already closed!
any ideas?

Can you give some more context to the problem?

 

[AD1]

Ok,

I'm using Visual Studio c#, here is the code which was stolen from an earlier thread

static void Main(string[] args)
{
IConnectionFactory factory = new NMSConnectionFactory(new Uri("stomp:tcp://datafeeds.networkrail.co.uk:61618"));

IConnection connection = factory.CreateConnection("MyUserName", "MyPassword");
ISession session = connection.CreateSession();

IDestination destination = session.GetDestination("topic://TRAIN_MVT_ALL_TOC");
IMessageConsumer consumer = session.CreateConsumer(destination);


connection.Start();
consumer.Listener += new MessageListener(OnMessage);
Console.WriteLine("Consumer started, waiting for messages... (Press ENTER to stop.)");

Console.ReadLine();
connection.Close();

When running this application the console windows pops up and then I get the error message

Apache.NMS.Stomp.ConnectionClosedException was unhandled
HResult=-2146233088
Message=The connection is already closed!
Source=Apache.NMS.Stomp
StackTrace:
at Apache.NMS.Stomp.Connection.CheckConnected()
at Apache.NMS.Stomp.Connection.Oneway(Command command)
at Apache.NMS.Stomp.MessageConsumer.DoClose()
at Apache.NMS.Stomp.MessageConsumer.Close()
at Apache.NMS.Stomp.Session.CreateConsumer(IDestination destination, String selector, Boolean noLocal)
at Apache.NMS.Stomp.Session.CreateConsumer(IDestination destination)
at ConsoleApplication1.Program.Main(String[] args) in c:\Users\Darren\Documents\Visual Studio 2012\Projects\ConsoleApplication1\ConsoleApplication1\Program.cs:line 21
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException:

 

[gavco98uk]

everything looks fine with the code you posted, nothing jumps out on me. The only thing I can suggest is double check your username/password are correct, and you have enabled the feed you are trying to connect to.

For reference, there is a code sample located at http://wiki.openraildata.info/index.php/C-Sharp although it's probably the same one you are using.

 

[AD1]

everything looks fine with the code you posted, nothing jumps out on me. The only thing I can suggest is double check your username/password are correct, and you have enabled the feed you are trying to connect to.

For reference, there is a code sample located at http://wiki.openraildata.info/index.php/C-Sharp although it's probably the same one you are using.

Cheers it was a password issue?

 

[headshot119]

I'm trying to receive messages from the train movement feed, using a PHP script.

The code is what's in the developers pack, run on my local machine using XAMPP, it keeps throwing a username or password error, even though they are correct.

Has anyone successfully written a PHP based receiver for the data feeds? Or does anyone have any suggestions as to what the problem could be ?

 

[Poggs]

OK, a few things to check:

1. When you log in to https://datafeeds.networkrail.co.uk/, do you see your user status as 'Active'?

2. Click on 'My Feeds' - click through the feed tabs (Train Movements, TD, VSTP, TSR...) - have you subscribed to any feeds? If not, click the feed name on the left, select one or more feeds in that category, and 'Update Subscription'

3. Check that the topic you're trying to subscribe to is prefixed with /topic, e.g. /topic/TRAIN_MVT_ALL_TOC for the all-TOC train movements (TRUST) feed

If you're still having trouble, drop an email to [email protected] and one of my colleagues will be able to help you further.

 

[headshot119]

Poggs, after reading through the first couple of pages of the thread I've resolved the issue, although I can receive the odd message from the feed now with the PHP receiver, it seems no where near as reliable as my Ruby implementation, and likes to timeout all over the shop.

I might have to look into something in .NET to achieve what I want.

 

[Poggs]

Poggs, after reading through the first couple of pages of the thread I've resolved the issue, although I can receive the odd message from the feed now with the PHP receiver, it seems no where near as reliable as my Ruby implementation, and likes to timeout all over the shop.

I might have to look into something in .NET to achieve what I want.

Hmm - I don't see any reason why the PHP Stomp consumer wouldn't work.

Can you drop me a PM with the code you're using?

 

[D1009]

I see freight trains have started appearing on Realtime Trains, though showing as FRGT rather than a headcode. Is this by acident or design?

 

[Tom]

Design.

 

[TaysideTrainz]

I'm having a few problems using the data feeds via Network Rail. I have subscribed to a few feeds, but how do I get them?

 

[dooton]

I'm having a few problems using the data feeds via Network Rail. I have subscribed to a few feeds, but how do I get them?

Your going to need to be more specific, do you mean you've subscribed via the website?

If so you'll need to write a Stomp client in the programming language of your choice, many of which are detailed earlier in this thread. For my android app i've written one in Java which has been chugging away and updating an SQL database for months now. I can't directly give you the code at the moment as its for a Final Year University project, but i'd be more than willing to assist should you chose the java route. Unfortunately I have little to no ruby skills, and PHP doesn't seem to working great for the guys trying it.

Drop me a PM if you'd like

**UPDATE**
Forgot to mention, have a look at the developer PDF that Network Rail have, there are a number of simple Stomp Client implementations for you to have a play around with I can't remember where it's at on the net, but here's a link to my copy http://cl.ly/052V0G30321u

 

[Poggs]

I'm having a few problems using the data feeds via Network Rail. I have subscribed to a few feeds, but how do I get them?

Have a look at the wiki at http://nrodwiki.rockshore.net/ - there's instructions, examples and a whole load of information on there

 

[McDeathUK]

Hi Folks.. am I glad to find this forum about the NR feeds. I started looking into this yesterday and imagine my horror when i found out just how bad the documentation was.. cut and pastes from other documents.

I have installed STOMP on my test Debian server and am showing the following on a PHP info

Stomp	enabled
API version	1.0.5
SSL Support	enabled

Directive	Local Value	Master Value
stomp.default_broker	tcp://localhost:61613	tcp://localhost:61613
stomp.default_connection_timeout_sec	2	2
stomp.default_connection_timeout_usec	0	0
stomp.default_read_timeout_sec	2	2
stomp.default_read_timeout_usec	0	0

My question is this, and forgive the basic, noobish deivery. If STOMP is doing the reading from the NR feed, do I need to also install ActiveMQ on the server. I was browing the wiki and it sems to indicate this is not required.

Also, I see when I login I have a security token.. i assume the user name and password required in the example are my email address for the username and the security token for the password?

I try the example below

<?php
// Network Rail Stomp Handler example by ian13
$server = "tcp://datafeeds.networkrail.co.uk:61618";
$user = "username";
$password = "password";
$channel = "TRAIN_MVT_ALL_TOC";

$con = new Stomp($server, $user, $password);
if (!$con) {
   die('Connection failed: ' . stomp_connect_error());
}
$con->subscribe("/topic/" . $channel);
while($con){
   if ($con->hasFrame()){
       $msg = $con->readFrame();
       foreach (json_decode($msg->body) as $event) {
         // do stuff with $event here
       }
       $con->ack($msg);
   }
}
die('Connection lost: ' . time());
?>

and am just getting ERROR 500.

any starting advice is greatly appreciated

 

[Zoe]

If STOMP is doing the reading from the NR feed, do I need to also install ActiveMQ on the server. I was browing the wiki and it sems to indicate this is not required.

That is correct, there is no need to install ActiveMQ.

Also, I see when I login I have a security token.. i assume the user name and password required in the example are my email address for the username and the security token for the password?

It's your email address and the same password you use for the website.

 

[McDeathUK]

That is correct, there is no need to install ActiveMQ.


It's your email address and the same password you use for the website.

OK, so I put my user name and password in... now it takes a little longer but ultimatly STILL gives Error 500!

Not a good day =)

 

[Zoe]

OK, so I put my user name and password in... now it takes a little longer but ultimatly STILL gives Error 500!

I initially thought that you just wanted to use STOMP to connect your server to the feeds but having just re-read your post, you say you have installed STOMP on the server and are expecting that to handle the messages. STOMP is a protocol and not something you install so what exactly have you actually installed?

 

[McDeathUK]

I initially thought that you just wanted to use STOMP to connect your server to the feeds but having just re-read your post, you say you have installed STOMP on the server and are expecting that to handle the messages. STOMP is a protocol and not something you install so what exactly have you actually installed?

I have used the PECL STOMP package. To install, I typed "pecl install stomp" and made sure the stomp.so was in my php5.ini file and restarted apache.

A phpinfo shows the output above, that STOMP is running. I have also tried with the second example, and uploaded the stomp files to the directory where I am running the example from.

Pretty much followed this (both attempted)

http://nrodwiki.rockshore.net/index.php/PHP_Example

 

[McDeathUK]

Zoe, I am using the method described in the wiki "This example requires PECL STOMP package. To install, type "pecl install stomp"."

Found at http://nrodwiki.rockshore.net/index.php/PHP_Example

++++ more info +++++

OK, I am using the PECL Stomp installation on my Debian server as describe above, and after some experimentation I am making the initial connection.

Using this little snippet

<?php

/* connection */
$link = stomp_connect('tcp://datafeeds.networkrail.co.uk:61618','myusername', 'mypassword');

/* check connection */
if (!$link) {
    die('Connection failed: ' . stomp_connect_error());
} else {
	echo 'Connected';
}

/* close connection */
stomp_close($link);

?>

I get "connected" on the screen. If I change the password, username, server it fails.. good start

But if I try any kind of subscription, it fails with a server 500. I have tried various topics, though the one I am interested in is just the RTPPM_ALL

Things I have checked

sockets are enabled for php
i am subscribed to all feeds

has anyone done an RTPPM project I can run on my various servers to see if I can work out what is going wrong. The problem with a server 500 error is it very difficult to track down what is wrong

 

[Zoe]

What code are you using to subscribe?

 

[McDeathUK]

What code are you using to subscribe?

OK officially flummoxed.. sure i tried this version, however it worked. Here is my completed code

<?php
	$server = "tcp://datafeeds.networkrail.co.uk:61618";
	$user = "myemail";
	$password = "mypassword";
	$channel = "RTPPM_ALL";
	
	$con = new Stomp($server, $user, $password);
	if (!$con) {
	   die('Connection failed: ' . stomp_connect_error());
	}
	
	$con->subscribe("/topic/" . $channel);

	// receive a message from the queue
	$msg = $con->readFrame();
	
	// do what you want with the message
	if ( $msg != null) {
		echo "Received message with body '$msg->body'\n";
		// mark the message as received in the queue
		$con->ack($msg);
	} else {
		echo "Failed to receive a message\n";
	}
	
	die('Connection lost: ' . time());
?>

and it worked.. ah well, first barrier broken. Thanks for the speedy Responses! Good to know there is support and other devs out there to bounce off.

Now just to do some interesting things with the wall of text that has been delivered!
--- old post above --- --- new post below ---



*************************


More often than not, I am not getting any data back from the feeds (php / stomp).. i refresh and once in a blue moon I get the data. This is on most of the feeds.

IN the last few hours whilst coding, I have only had two requests successfully pull off some data

I am assuming the subscription is timing out to early.. is there any way of keeping the request going till it pulls some data off or is there something else I am missing?

<?php
	$server = "tcp://datafeeds.networkrail.co.uk:61618";
	$user = "myusername";
	$password = "mypassword";
	$channel = "RTPPM_ALL";
	
	$con = new Stomp($server, $user, $password);
	if (!$con) {
	   die('Connection failed: ' . stomp_connect_error());
	}
	
	$con->subscribe("/topic/" . $channel);

	// receive a message from the queue
	$msg = $con->readFrame();
	
	// do what you want with the message
	if ( $msg != null) {
		listPosts($msg->body);
		$con->ack($msg);
	} else {
		echo "Failed to receive a message\n";
	}
	
	die('Connection lost: ' . time());
?>

listPosts is just a function I am writing to process the data, but not getting that far which is making testing close to impossible

 

[Poggs]

You say you're getting a 500 error code... how are you running this script?

Are you running it through a web browser? If so, don't - it's a long-running process and you should run it through the command line. I have a hunch that the 500 errors you're getting are from your server, not from the Datafeeds platform.

 

[McDeathUK]

The double post above confused things... sorry.. merged my replies

I have got the code working now and am no longer getting 500 error and yeah you are spot on (and I never thought is was the datafeed) the issue was defiantly 100% my code or my server., but now I am more often than not failing to get a message. Literally in the last 3 hours I have had 3 successful messages come though on various feeds, but the failure or success is within seconds of reload

As the attempt never seems to be for very long, so I am thinking the request is timing out too fast.

For testing purposes, yeah I am running it through a browser

 

[Poggs]

As the attempt never seems to be for very long, so I am thinking the request is timing out too fast.

For testing purposes, yeah I am running it through a browser

Try *not* running it through a browser - see http://php.net/manual/en/features.commandline.php.

There are all sorts of problems you might get with running a long-lived daemon under Apache, such as the server buffering data before sending it to your browser (so you think you've not received messages) that will only confuse matters.

 

[McDeathUK]

Try *not* running it through a browser - see http://php.net/manual/en/features.commandline.php.

There are all sorts of problems you might get with running a long-lived daemon under Apache, such as the server buffering data before sending it to your browser (so you think you've not received messages) that will only confuse matters.

I tried running through a command line on the server console but am getting

PHP Fatal error: Class 'Stomp' not found in /(path)/ppmdata.php on line 7

I have installed Stomp from PECL and it is showing in my php info file as running

 

[McDeathUK]

Try *not* running it through a browser - see http://php.net/manual/en/features.commandline.php.

There are all sorts of problems you might get with running a long-lived daemon under Apache, such as the server buffering data before sending it to your browser (so you think you've not received messages) that will only confuse matters.

OK, not sure if my older posts will still arrive.. but I have got Stomp working on the command line now by using the package from the stomp site and putting that in my root. I am using the code by ian13 from the wiki (file just happens to be called first.php).. configured with my setings.. but still getting error 500.. but with more info

PHP Fatal error: Uncaught exception 'StompException' with message 'Socket connection hasn't been established' in /www/development.xxx/public/trainfeeds/Stomp.php:500
Stack trace:
#0 /www/development.xxx/public/trainfeeds/Stomp.php(347): Stomp->_writeFrame(Object(StompFrame))
#1 /www/development.xxx/public/trainfeeds/first.php(15): Stomp->subscribe('/topic/RTPPM_AL...')
#2 {main}
thrown in /www/development.xxx/public/trainfeeds/Stomp.php on line 500
--- old post above --- --- new post below ---
Sorted. I went down the node.js route, and my server is now churning out PPM data like a champ!