Is this a serious suggestion? How would you expect anyone to travel from St James Park to Axminster? Or Hartford to Warrington Bank Quay? Or a hundred other examples?
Yes. I am serious. In this case two tickets should be offered, because by doubling back, you are travelling further than the destination of your ticket which isn't something normally allowed. If you sometimes allow it, it will just cause confusions.
Practically, the ticket machine should give a warning why such a ticket is not available, as someone trying to find a route in the route map will not expect to travel further than the destination.
For example, if I walk into Luton station and want to travel to Kentish Town, I will look at the route map first, which shows a direct route. If there is a train at the departure board which calls at Kentish Town, great. If not, I will cross-reference the calling point with the route map to see if I can change at an intermediate station, for example, St Albans or West Hampstead Thameslink.
The process is exactly the same as travelling on the London Underground using a tube map, where double-backing is not a normal circumstances and is only used when engineering works or disruptions render certain platforms unavailable.