Trainline "AI" assistant: incorrect advice

[MrJeeves]

Trainline has launched a wonderful new "AI" assistant feature after you purchase a ticket from their app(/site?).

One of the first things I asked it, after buying a child ticket, was whether I could travel on it as a 28 year old adult. Unsurprisingly, the answer was... yes.

Me: Does that mean I can travel on this ticket as a 28 year old?

"AI": Yes, absolutely! As a 28-year-old, you can definitely travel on this Off-Peak Day Return ticket between Burgess Hill and Wivelsfield. This ticket is valid for adult passengers. Just remember to adhere to the off-peak travel times and the specific date on your ticket (Sun 8 Jun 2025). Your outbound journey is scheduled for 20:43 from Burgess Hill to Wivelsfield, and your return is at 21:33 from Wivelsfield to Burgess Hill. Is there anything else you'd like to know about your journey or ticket?

I then went on to ask about time restrictions on my ticket, which TTL have decided to omit from whatever information bank the agent is provided with as it simply informs be about "general" off-peak timings (after 0930).

I also tried to process a refund via the chatbot. It first broke the bot and told me it was now "unavailable" as something went wrong. The second (and subequent) times, it told me that a refund had been processed. It hadn't.

I see this as a minefield for potential future problems for passengers, and really should never have been done. Humans struggle to understand ticketing restrictions a lot of the time, let alone a large language model that isn't particularly specialised, nor has sufficient data to answer real questions.

 

[30907]

Oh dear!

Though TBF there are no time restrictions on Sunday so that isn't incorrect.

 

[Nova1]

I'd love to see this tried as authority to travel / itinery

 

[Egg Centric]

That's shockingly bad.

The sad thing is it shouldn't have been too difficult to create an infinitely better bot (even using a stock LLM). It shows a total lack of expertise and makes you wonder about their QA processes.

 

[Trainguy34]

Quite ironic that one of the stations in the example has a code of "BUG" considering there must be one in the code.

 

[sheff1]

I'd love to see this tried as authority to travel / itinery

Seems a clear authorisation from a Licensed Retailer, on behalf of a Train Company.

 

[styles]

You've got to be extremely confident in your LLM to let it loose answering questions about contractual terms and conditions created by a third-party.

Why anyone would do this, having presumably seen the memes about tricking ChatGPT into saying 2+2=7 and that there's only 4 colours in the rainbow, I've no idea.

This is just an awful idea which I'm betting they'll have to roll back on at some point when they see the complaints come in.

Either that or they'll write a series of hacks to account for the long list of errors; at which point you'd have been better off writing a "traditional" chatbot where the goal is basically to route a user to a predefined answer.

 

[signed]

I'd love to see this tried as authority to travel / itinery

Not in the UK, but in Canada, Air Canada did that kind of thing with LLM and the customer won their lawsuit claming the LLM was contractual agreements

== Doublepost prevention - post automatically merged: Monday at 06:33 ==

Artificial intelligence is having a growing impact on the way we travel, and a remarkable new case shows what AI-powered chatbots can get wrong – and who should pay. In 2022, Air Canada's chatbot promised a discount that wasn't available to passenger Jake Moffatt, who was assured that he could book a full-fare flight for his grandmother's funeral and then apply for a bereavement fare after the fact.
According to a civil-resolutions tribunal decision last Wednesday, when Moffatt applied for the discount, the airline said the chatbot had been wrong – the request needed to be submitted before the flight – and it wouldn't offer the discount. Instead, the airline said the chatbot was a "separate legal entity that is responsible for its own actions". Air Canada argued that Moffatt should have gone to the link provided by the chatbot, where he would have seen the correct policy.

Airline held liable for its chatbot giving passenger bad advice - what this means for travellers

When Air Canada’s chatbot gave incorrect information to a traveller, the airline argued its chatbot is "responsible for its own actions".

www.bbc.com

 

[island]

And this is why the AI fad should calm down.

 

[Kilopylae]

I'd love to see this tried as authority to travel / itinery

Savvy and unscrupulous would-be fare evaders are almost certainly going to try tricking it into giving them the answer they want; the question is whether they'll know enough (about itineraries from an accredited retailer etc) to stick with that story after being told off. We'll probably hear about it here in a few weeks.

 

[enyoueffsea]

Have they not even been savvy enough to add a caveat in their T&Cs to completely absolve them of any errors in the advice?

 

[Bletchleyite]

And this is why the AI fad should calm down.

AI is certainly absolutely unsuitable for anything involving public transport information provision. At most, it could be used to make a chatbot sound more human, but the actual information needs to be fact, not inferred.

I've long defended Trainline from accusations from staff that it provides information different from other retailers - to now it couldn't and doesn't as it uses basically the same data sources as everyone else (and any cases where it doesn't also affect the Trainline run TOC sites). But this is just absolute nonsense and will cause all manner of problems - very irresponsible.

 

[azzafazza1234]

Have they not even been savvy enough to add a caveat in their T&Cs to completely absolve them of any errors in the advice?

That would be a show of faith!

Terms and Conditions | Trainline

Read the terms and conditions for thetrainline.com website.

www.thetrainline.com

Image of conditions 11, Liability, of Trainline's terms and conditions (linked above) with the following sentence highlighted:
We are also not responsible for any losses that you suffer that are related to your use of the AI Assistant.

 

[Watershed]

That would be a show of faith!

Terms and Conditions | Trainline

Read the terms and conditions for thetrainline.com website.

www.thetrainline.com

View attachment 181634

Such purported boilerplate exclusions are highly unlikely to hold water in Court, especially given the context of a consumer contract. There is an implied term that the service will be provided with reasonable care and skill under the Consumer Rights Act 2015, which a trader cannot exclude.

 

[Bletchleyite]

Such purported boilerplate exclusions are highly unlikely to hold water in Court, especially given the context of a consumer contract. There is an implied term that the service will be provided with reasonable care and skill under the Consumer Rights Act 2015, which a trader cannot exclude.

And I think it's fair to say, based on how they work, that anyone providing public transport information by way of generative AI (rather than simply using gen AI as the user interface) is not providing the service with reasonable care and skill. The nature of gen-AI is such that it cannot be guaranteed to provide correct answers to black-and-white questions - it simply does not work that way.

 

[island]

And I think it's fair to say, based on how they work, that anyone providing public transport information by way of generative AI (rather than simply using gen AI as the user interface) is not providing the service with reasonable care and skill. The nature of gen-AI is such that it cannot be guaranteed to provide correct answers to black-and-white questions - it simply does not work that way.

Such purported boilerplate exclusions are highly unlikely to hold water in Court, especially given the context of a consumer contract. There is an implied term that the service will be provided with reasonable care and skill under the Consumer Rights Act 2015, which a trader cannot exclude.

However, information such as this that has been provided after a contract has been concluded won't form part of the contract. Nor will it form a new contract, as no consideration has been given.

If one wanted to be able to rely on whatever nonsense the AI put out, it would be necessary, at least, to have that conversation prior to purchase.

 

[Watershed]

However, information such as this that has been provided after a contract has been concluded won't form part of the contract. Nor will it form a new contract, as no consideration has been given.

If one wanted to be able to rely on whatever nonsense the AI put out, it would be necessary, at least, to have that conversation prior to purchase.

Post-purchase information could absolutely become a term of the contract, under section 50(1)(b) of the Act:

(1) Every contract to supply a service is to be treated as including as a term of the contract anything that is said or written to the consumer, by or on behalf of the trader, about the trader or the service, if—

(a) it is taken into account by the consumer when deciding to enter into the contract, or

(b) it is taken into account by the consumer when making any decision about the service after entering into the contract.

 

[Egg Centric]

Does the UI have anything at the beginning of chat saying "don't rely on me" in big letters or anything like that?

 

[MrJeeves]

Does the UI have anything at the beginning of chat saying "don't rely on me" in big letters or anything like that?

There is a bit that says it may provide "interesting" answers, and that's all really.

 

[Krokodil]

I'd love to see this tried as authority to travel / itinery

My patience gets exhausted enough trying to argue with organics who have no clue. Throwing AI slop into the mix will probably give me a stroke!

I've long defended Trainline from accusations from staff that it provides information different from other retailers - to now it couldn't and doesn't as it uses basically the same data sources

Like when it offers an "Open Return" tab that will actually list every sort of day and period return existing?

 

[Egg Centric]

There is a bit that says it may provide "interesting" answers, and that's all really.

Ta.

They seem to have implemented this completely back to front. I started typing out how they should have implemented it (unlike other posters here I believe it would actually be quite easy to implement for this use case when like Trainline you already have access to a ticket validation engine) but it's off topic and I don't see why I should help them for free!

 

[Adam Williams]

Why anyone would do this, having presumably seen the memes about tricking ChatGPT into saying 2+2=7 and that there's only 4 colours in the rainbow, I've no idea.

AI (and installing execs who are solely there to shoehorn AI into the product) make stock prices go brrrrr

For now, anyway.

 

[OscarH]

AI (and installing execs who are solely there to shoehorn AI into the product) make stock prices go brrrrr

For now, anyway.

It'll burst eventually, but that'll be fine, they'll just do some more short-sighted mass layoffs, the shareholders won't suffer

 

[duffield]

Have they not even been savvy enough to add a caveat in their T&Cs to completely absolve them of any errors in the advice?

If they did, I've got a feeling that if a case came to court, the court would take a very dim view of that tactic:

"Here's our helpful AI chatbot to give you advice about your ticket!" **

** Any advice given may be complete nonsense, and if you follow it you may be liable for an excess fare, a penalty fare or prosecution.

Disclaimers like this - as I understand it - are only legally valid under consumer law if they are considered fair and reasonable.

 

[Adam Williams]

This ticket had a 16-25 Railcard applied. I paid £0.85 for it.

Downright irresponsible.

 

[superkopite]

How does one access it? I cannot see it on my app

 

[duffield]

How does one access it? I cannot see it on my app

With any luck they've been reading this thread and have withdrawn it before its obvious defects become too much of a liability.

 

[superkopite]

With any luck they've been reading this thread and have withdrawn it before its obvious defects become too much of a liability.

Nope, no such luck

I found it, you need to buy a ticket first!

 

[nwales58]

We don't know exactly what this AI assistant is but ...

WHY on earth would you use a LLM, which is general-purpose, behind a robot working in a specific domain such as GB rail ticketing???

This domain has specific rules, admittedly weird in places, validation requirements for retailing, a knowledgebase and so on. Surely you train the model on that body of data. Then it should spew out the relevant restriction (in human-speak such as 'you can use any train on a Sunday') and totally understand passenger types adult/child/railcard/combinations.

The management decisions behind procuring and releasing this stuff baffles me.

 

[Bletchleyite]

The management decisions behind procuring and releasing this stuff baffles me.

It's a pure game of buzzword bingo.