• Our new ticketing site is now live! Using either this or the original site (both powered by TrainSplit) helps support the running of the forum with every ticket purchase! Find out more and ask any questions/give us feedback in this thread!

does any one else on here use raildata.org.uk to get NWR Train Describer (TD) data

Trains1

Member
Joined
7 Aug 2024
Messages
120
Location
manchester
Hello,

I am trying to use TD data from raildata.org.uk. I had it working a few weeks ago, but now that I've come back to it, it no longer works.

I am using Python—does anyone else use it via Python? If so, does it still work for getting live signal data?
 
Sponsor Post - registered members do not see these adverts; click here to register, or click here to log in
R

RailUK Forums

Vexed

Member
Joined
12 Jan 2020
Messages
309
Location
Herts / Hants

Trains1

Member
Joined
7 Aug 2024
Messages
120
Location
manchester
No worries, here's a Gist of what I do. It's adapted from the example provided in the (hard to find and slightly out of data as it doesn't mention groups at all!) PubSub documentation.

Great thank you, I'll have a look at it tomorrow

== Doublepost prevention - post automatically merged: ==

apparently there may be a Publisher issue
 
Last edited:

Vexed

Member
Joined
12 Jan 2020
Messages
309
Location
Herts / Hants
apparently there may be a Publisher issue
It took me about 2 months of back and forth with support before I managed to get it to work, but that was an issue affecting most (all?) consumers so shouldn't happen in your case.

It is worth reaching out to support if you're sure everything's correct your end.
 

arb

Member
Joined
31 Oct 2010
Messages
498
Inspired by this thread and the example code (thanks!) I decided to try registering with the rail data marketplace to see if I could find anything interesting to do with any of the data they provide.

I'm struggling to get the code example from @Vexed to work, with "Authentication Failed" errors. Can I double-check exactly what you're putting in the three empty variables in the code you shared, before I try contacting their support?
  • For CONSUMER_GROUP I'm using the value given in the Access Credentials section of the Pub/Sub page for the feed I've subscribed to.
  • For CONSUMER_USERNAME and CONSUMER_PASSWORD I've been using my login details for the raildata.org.uk website itself.
The PDF that you linked to (which you said yourself was out-of-date, so this might not be relevant) talks about a consumer key and secret which I can't seem to find anywhere. Are they what I should be using for the username and password instead of the website login details?
 

Trains1

Member
Joined
7 Aug 2024
Messages
120
Location
manchester
Inspired by this thread and the example code (thanks!) I decided to try registering with the rail data marketplace to see if I could find anything interesting to do with any of the data they provide.

I'm struggling to get the code example from @Vexed to work, with "Authentication Failed" errors. Can I double-check exactly what you're putting in the three empty variables in the code you shared, before I try contacting their support?
  • For CONSUMER_GROUP I'm using the value given in the Access Credentials section of the Pub/Sub page for the feed I've subscribed to.
  • For CONSUMER_USERNAME and CONSUMER_PASSWORD I've been using my login details for the raildata.org.uk website itself.
The PDF that you linked to (which you said yourself was out-of-date, so this might not be relevant) talks about a consumer key and secret which I can't seem to find anywhere. Are they what I should be using for the username and password instead of the website login details?
from my understanding, there should be on the Pub/Sub page a Consumer group, Consumer username, and Consumer password these are the ones that I've use and it worked but seemed to stop working with a different error
 

arb

Member
Joined
31 Oct 2010
Messages
498
I only have a consumer group. Not a consumer username or password.
 

Vexed

Member
Joined
12 Jan 2020
Messages
309
Location
Herts / Hants
Are they what I should be using for the username and password instead of the website login details?
from my understanding, there should be on the Pub/Sub page a Consumer group, Consumer username, and Consumer password these are the ones that I've use and it worked but seemed to stop working with a different error
Yes, once subscribed you should be using the three keys under the Pub/Sub section on the data product page.

1739989394804.png

I only have a consumer group. Not a consumer username or password.
If any of this is missing I would suggest contating support.
 

arb

Member
Joined
31 Oct 2010
Messages
498
Thanks, at least now I have quite a specific problem to raise with them.
 

Vexed

Member
Joined
12 Jan 2020
Messages
309
Location
Herts / Hants
Thanks, at least now I have quite a specific problem to raise with them.
Just a heads up they can be quite slow to reply - my issue was that there was no consumer group listed. I now have an interesting situation where I'm still using the consumer group they told me to use instead of the one on the dashboard but it still works so... Anyway I digress. If you don't hear back within 2-3 weeks you could ask on the open rail data Google Group for further guidance. It's not a great situation and not what the future open data should be like, hopefully the support situation has improved over the last few months since I last had to use it.

Just out of curiosity, when did you subscribe to the NWR TD data product?
 

arb

Member
Joined
31 Oct 2010
Messages
498
I subscribed just a few days ago. I left it for a little while in case the authentication issues just needed a bit of time for something to sync across their systems, but still no luck as of today. I've just submitted the support request so we'll see how long it takes them to respond. I didn't know about the google group - am reading through some of the threads there now.

== Doublepost prevention - post automatically merged: ==

--- below is a separate post which the forum has merged into the previous one ---

I got a response to my support request first thing this morning. They say there is a known issue with pub/sub access and they intend to deploy a fix over the weekend.

== Doublepost prevention - post automatically merged: ==

--- another update below which I guess the forum will merge into the previous post again ---

Update on Monday morning: I now have a username and password and get beyond the authentication error. (Also my BOOTSTRAP_SERVER has been changed, but not my CONSUMER_GROUP). The new problem, though, is that no data is ever returned to me. I now seem to be in pretty much the same position as this thread in the google group you referred to.

Regarding the example code you shared, I'm not sure where you got your TOPIC_NAME (starting "prod-1010...") from? If I use that, I get a different error, "Subscribed topic not available". Using a name like "TD_ALL_SIG_AREA" fixes that problem, but, as I said above, no data is received.
 
Last edited:

Vexed

Member
Joined
12 Jan 2020
Messages
309
Location
Herts / Hants
Update on Monday morning: I now have a username and password and get beyond the authentication error. (Also my BOOTSTRAP_SERVER has been changed, but not my CONSUMER_GROUP). The new problem, though, is that no data is ever returned to me. I now seem to be in pretty much the same position as this thread in the google group you referred to.
Everyone's connection details changed, I haven't got around to updating mine yet. I'll do it later and let you know if it works.

Connecting but getting no data is very annoying, I know the feeling! This might sound odd, but when I was trying to figure this out last year I got Kafka error messages on Ubuntu that were silent on Windows. Does this help?

Regarding the example code you shared, I'm not sure where you got your TOPIC_NAME (starting "prod-1010...") from? If I use that, I get a different error, "Subscribed topic not available". Using a name like "TD_ALL_SIG_AREA" fixes that problem, but, as I said above, no data is received.
"prod-1010..." is a topic for Darwin data.
 

Top