Can computer modelling be used in the testing and designing of timetables on a large scale?

[Ianno87]

It looks like the kind of thing which would be a good collaboration with a Computer Science department at a University, which would substantially reduce the cost.

What makes you think that is not already happening? I've reviewed some myself in my time. Some lovely ideas, but struggle to stand up to scrutiny when applied to reality.

 

[OxtedL]

I work in a different industry but where we also have to manage big complicated changeovers and where computer modelling is highly relevant.
These views are my own and not those of my employer.

Well with enough computing power you could try and Montecarlo it.

Montecarlo would be an incredibly bad way to generate a national rail timetable. Even with the best available computers you would need a world-endingly large number of simulations before you stumbled across a solution that had even half of trains planned within timetabling rules. Let alone something as well-optimised as what is currently produced by "hand".

If you wanted to use computers to generate a national rail timetable, you would need to use some very sophisticated and bespoke machine learning techniques.
Nothing off the shelf would work as while world leaders in this area like e.g. the airline industry and Amazon will have made lots of headway into the difficult problems, the railway has whole orders of magnitude more constraints to work around. This is a field where complexity grows exponentially with the number of constraints you add.

So we likely need years of R&D with the finest brains available before we are even close to ready to replace the current process.
A university research project would not cut it. You need a large team of experts working full time on nothing else.

Would we want to use a machine generated timetable though?
e.g. if the new magic timetable generator allowed us to update the timetable every 2 months instead of every 6, how would you feel telling passengers in Oxted that their train times in the new timetable had changed beyond recognition to allow a new freight path through the Peak District? And that they would likely change beyond recognition again in another couple of months for a similarly trivial reason?

There are a lot of features of timetable changes we take for granted now (like most times staying the same if you live in an area which isn't being recast, or that some trains can't be moved 10 minutes later else they become useless to commuters starting at 9am, etc) that we don't even think of as timetabling constraints and would make generating a good quality solution very hard.

Transport is still a retail industry. Teaching our timetable generator about the quality of a timetable for end users is a whole other level of intractability above just finding a workable solution. So if I worked in the train planning, I would not be especially worried about a computer replacing me just yet.

So what is possible to reduce the time it takes to produce a timetable?
The number of different elements involved and the number of different kind of changes you could make to the timetable are large enough that any substantial automation of the existing process is still very very hard, even accepting a magic timetable generator is unrealistic.

Speaking from experience in my own industry, and at the risk of looking foolish by not knowing enough about current timetabling practices, I suspect the biggest opportunities to shorten critical path will lie in:

• Relentlessly pursuing all the simple automation you can (e.g. checking train diagrams are legal). It sounds like NR have done lots of this. It sounds like TOCs may be less consistent.
• Making sure the whole industry is working from the same platform and with the same capability. e.g. working via email/telephone gets things done and gets things done well. But it's too informal if you need to regularly and quickly turn around something highly technical with lots of different people needing to approve it.

I suspect the industry is a long way away from exhausting the possibilities of these two options. And doesn't need to buy in huge amounts of AI expertise from silicon valley just yet.

 

[6Gman]

When it gets to the stage of the argument where claims are made that progression is impossible because the people who are currently undertaking the task are completely overwhelmed by their workload, then it's time to admit the current method is no longer fit for purpose.

But there is a fair point to be made that if a substantial proportion of the train planning staff are to spend months (years?) detached to develop, inform and modify - as necessary - a new system who is going to put the timetable together in their absence?

 

[PeterC]

But there is a fair point to be made that if a substantial proportion of the train planning staff are to spend months (years?) detached to develop, inform and modify - as necessary - a new system who is going to put the timetable together in their absence?

I had this issue, fortunately on a scale many orders of magnitude lower, on a non railway site. The brutal answer is that you have a freeze, or at least downgrade the importance of some work. If that means skipping a timetable change then so be it.

You either accept that change will hurt or you carry on struggling.

 

[HSTEd]

Montecarlo would be an incredibly bad way to generate a national rail timetable. Even with the best available computers you would need a world-endingly large number of simulations before you stumbled across a solution that had even half of trains planned within timetabling rules. Let alone something as well-optimised as what is currently produced by "hand".

If you wanted to use computers to generate a national rail timetable, you would need to use some very sophisticated and bespoke machine learning techniques.

The problem is that we don't have anything like the training set to do a brute force machine learning simulation.
I dont think enough national timetables will exist to do it properly. Or with enough variation in the network to prevent it rote learning the process applied by planners to the current network!

The approach almost certainly has to be from first principles.

I'm not suggestingly randomly generating the entire timetable and then determining whether or not it is a valid timetable or not - that is clearly an absurd approach.
I was talking about selecting which trains should go into teh process "first" in the queue and thus recieve first dibs on paths.

But the trend in British railway operations is towards a clockface railway anyway, which substantially cuts the decision space.
A clockface hourly service has only 60 possible times it can leave a station after all(and you get a dozen+ trains with that one decision), a half hourly service has only 30 times etc etc etc.

The interesting question that engenders is - how many stopping patterns are in use?

e.g. if the new magic timetable generator allowed us to update the timetable every 2 months instead of every 6, how would you feel telling passengers in Oxted that their train times in the new timetable had changed beyond recognition to allow a new freight path through the Peak District? And that they would likely change beyond recognition again in another couple of months for a similarly trivial reason?

A timetable generator that could run quickly would enable infrastructure modfications to be tested as to their impact on the overall timetable in a way that they simply can't today.

Does adding more track to this place jiggle the timetable to allow more trains to run someplace else for example?

And once the model is built, the gradually reducing price of processing power would enable it to be used more and more widely over time.

Until we have masters students using it to justify a case study on adding a new track from place x to y!

 

[Ianno87]

I had this issue, fortunately on a scale many orders of magnitude lower, on a non railway site. The brutal answer is that you have a freeze, or at least downgrade the importance of some work. If that means skipping a timetable change then so be it.

You either accept that change will hurt or you carry on struggling.

Given the time and effort for into major timetables, it'd be more like suspending any ability to change the timetable for 5 years.

Which is a great way to lose irreplacable skills and knowledge, that will be needed to sense check the inplememtation of any system.

 

[EllieAquitrain]

When it gets to the stage of the argument where claims are made that progression is impossible because the people who are currently undertaking the task are completely overwhelmed by their workload, then it's time to admit the current method is no longer fit for purpose.

But sometimes we decide that long-term benefit is not worth short-term pain. This is something I'd think would be well-understood by someone on a rail forum! Deliberately using an extreme example just to make the point that it's *possible* for it to not be worth it, you wouldn't pay £900 million just to save a few trains an hour one minute in their journey (yes of course that could in theory have knock on benefits making it worth £900 million but in this hypothetical there are none!)

 

[Ianno87]

But sometimes we decide that long-term benefit is not worth short-term pain. This is something I'd think would be well-understood by someone on a rail forum! Deliberately using an extreme example just to make the point that it's *possible* for it to not be worth it, you wouldn't pay £900 million just to save a few trains an hour one minute in their journey (yes of course that could in theory have knock on benefits making it worth £900 million but in this hypothetical there are none!)

Especially when the long term benefit actually materialising in practice is a very long way from being certain.

 

[EllieAquitrain]

Especially when the long term benefit actually materialising in practice is a very long way from being certain.

Indeed. In this thread's case, there's not exactly any guarantee the AI will be better (or even usable at all), only the potential. I don't think the potential is likely enough to be achieved, nor of sufficient benefit over the current system to make it worthwhile.

 

[Dr Day]

Some genuine questions from a non-train planner, possibly going off topic so apologies in advance.

Most of this thread has been about the production or supply side of the timetable, although it isn't clear to me what specific problem we are trying to solve. How much of the 'problem' stems from the process itself? With today's technology for passengers to find out train times do we really still need two timetables a year with every single train being 'bid' to NR, validated and 'offered' back (which is my understanding of how it essentially works)? Could we have a rolling permanent timetable, with changes made when only required (either permanent or temporary)? Do changes which don't affect anyone else need to go into NR and back out again? Presumably this will be tested with the Cardiff Valleys no longer being part of NR's network, yet a desire to maintain through ticketing and national rail enquiries to/from stations on that part of the network.

What about the demand side? We are moving towards more clock-face timetables (which have certain benefits) and tend to use price to manage demand rather than run services when people necessarily want them (other than fairly blunt peak extras - be it time of day, day of week, special events, summer Saturdays etc). Is there a role for more passenger-demand data-driven timetabling?

Going back to the original thread, would a return to more centralised control mean a return to some of the systems development and R&D organisations that existed internally within BR, before they got privatised and sold to Sema/ATOS/AEA Technology etc? Would that lead to more 'fit for purpose' timetabling system, or suite of linked data-driven systems for the various elements from infrastructure capacity to day-to-day rosters to reservations/AP to performance analysis? Various optimisation tools already exist around say driver diagrams and could be applied to pathing but this is only part of a much wider picture across the industry which has 'the timetable' at its heart.

 

[Ianno87]

Some genuine questions from a non-train planner, possibly going off topic so apologies in advance.

Some good questions, definitely worth a response to!

Most of this thread has been about the production or supply side of the timetable, although it isn't clear to me what specific problem we are trying to solve. How much of the 'problem' stems from the process itself? With today's technology for passengers to find out train times do we really still need two timetables a year with every single train being 'bid' to NR, validated and 'offered' back (which is my understanding of how it essentially works)? Could we have a rolling permanent timetable, with changes made when only required (either permanent or temporary)? Do changes which don't affect anyone else need to go into NR and back out again? Presumably this will be tested with the Cardiff Valleys no longer being part of NR's network, yet a desire to maintain through ticketing and national rail enquiries to/from stations on that part of the network.

I guess the simple reason for fixed timetable change dates is the inter-relatedness of the system; call them "opportunities for controlled change"; i.e. if an operator wants to rectify a particular station dwell time, the timetable dates are the opportunity to manage all the required changes in one opportunity; doing one at a time more gradually might yield a more inefficient timetable solution than considering multiple together at the same time. The more you can build in together at the same time, the better, as there is ore opportunity to rethink the plan more extensively in one go, rather than fudging in one thing at a time (to put it crudely).

There have been precedents for significant timetables changes outside the usual two Windows; Chiltern and London Midland in (IIRC) September 2011, for example.

What about the demand side? We are moving towards more clock-face timetables (which have certain benefits) and tend to use price to manage demand rather than run services when people necessarily want them (other than fairly blunt peak extras - be it time of day, day of week, special events, summer Saturdays etc). Is there a role for more passenger-demand data-driven timetabling?

The up side of clock face timetables is that they, in general, drive efficiency in terms of rolling stock or train crew utilisation (for example). Passenger demand isn't the sole driver of how train services are designed. Political considerations play a part, as do revenue consideration (e.g. you may prioritise a train with fewer passengers, but more revenue, for example). The weighting given to each varies greatly depending on which part of the network you are on, time of day, day of week, direction of travel, etc. etc.

This is where it is probably easy to design something software to optimise around one or two things, but designing something that achieves the best compromise between *everything* (as well as a credible operational plan itself) is a significant order of magnitude more difficult. For example, how do you design software to optimise the most politically palatable train service?!

Going back to the original thread, would a return to more centralised control mean a return to some of the systems development and R&D organisations that existed internally within BR, before they got privatised and sold to Sema/ATOS/AEA Technology etc? Would that lead to more 'fit for purpose' timetabling system, or suite of linked data-driven systems for the various elements from infrastructure capacity to day-to-day rosters to reservations/AP to performance analysis? Various optimisation tools already exist around say driver diagrams and could be applied to pathing but this is only part of a much wider picture across the industry which has 'the timetable' at its heart.

However the "timetable" has a circular relationship to everything else.

Sometimes, you design the train service to meet demand, and live with the train crew consequences of that. e.g. you may accept an inefficient driver roster if it means you can run a very lucrative train.

Sometimes you design the train service around efficient train crew working, and the timetable is what it is (e.g. Whitby or Lincolnshire or Leeds-Lancaster).

Most of the network is some subjective, judgement-based compromise between the two.

 

[Dr Day]

This is where it is probably easy to design something software to optimise around one or two things, but designing something that achieves the best compromise between *everything* (as well as a credible operational plan itself) is a significant order of magnitude more difficult. For example, how do you design software to optimise the most politically palatable train service?!

Most of the network is some subjective, judgement-based compromise between the two.

Absolutely - and this is probably the nub of the issue. Most things can be boiled down to money, but even then the 'optimal' timetable for overall value for money for the Treasury probably wouldn't go down very well with anybody else (especially on this forum). Nor the 'optimal' in terms of decarbonisation with the Environment Secretary etc. But would be an interesting academic exercise.

Thanks

 

[HSTEd]

Absolutely - and this is probably the nub of the issue. Most things can be boiled down to money, but even then the 'optimal' timetable for overall value for money for the Treasury probably wouldn't go down very well with anybody else (especially on this forum). Nor the 'optimal' in terms of decarbonisation with the Environment Secretary etc. But would be an interesting academic exercise.

Thanks

I think a railway system model is potentially extraordinarily useful for precisely that reason.
If such a model exists, the potential costs and benefits of any number of different tradeofs becomes a simple exercise of running the model repeatedly with different input criteria.
That makes those tradeoffs clear in a way that they may not be now.

 

[bspahh]

I think a railway system model is potentially extraordinarily useful for precisely that reason.
If such a model exists, the potential costs and benefits of any number of different tradeofs becomes a simple exercise of running the model repeatedly with different input criteria.
That makes those tradeoffs clear in a way that they may not be now.

I know a bit about modeling, and very little about timetabling ...

There is a broad split between "black box" prediction methods which give good results, where the reasoning is next to impossible to understand, and methods which are easy for a human to interpret - either for an expert or ideally for a lay person.

If you have winners and losers from a decision, then you might end up having to pick a sub-optimal solution. That is a political problem, not a technical one.

Weather forecasts have become more accurate, as the background models get updated with live data. I will trust a prediction of when its going to rain, when its backed up with a radar image that shows a rain cloud is approaching. Long range forecasts collate a vast amount of measurements. There is still a role for a weather forecaster to combine these predictions with their skill and judgement.

Accurate modeling of the operations of a timetable would be useful for testing the resilience of the network. You could take an ideal setup, add in some random variations and then simulate it enough times to get good quality statistics. This might highlight weak spots which could be improved. You could set an accurate price for a freight train to discourage journeys at a time which would be most likely to cause disruption. This could also let you optimize how to recover when there has been a problem, although that would need a load of updated live data throughout the network.

 

[Bald Rick]

Could we have a rolling permanent timetable, with changes made when only required (either permanent or temporary)?

To add to the excellent reply of @Ianno87....

Timetable changes vary considerably in size, from a complete recast of several ‘related’ operators at one extreme to a slightly amended calling pattern on one service at the other. Even for the smallest changes, the revisions need to be ‘validated’ - essentially checked to make sure it doesn’t interfere with anything else in the timetable. Any conflicts identified then have to be resolved on a case by case basis. Clearly the bigger the proposed change, the bigger the validation exercise; sometimes there can be thousands of conflicts to resolve just for one operator.

By doing changes on specific dates (twice a year in our Network’s case), that validation is done on a network wide basis twice a year.

With a rolling timetable, if, for example, operator A decided to make a change on the July 6th, it would require validation against all other schedules that the change interacts with, and may require hundreds (or thousands, as above) of conflicts to be resolved. And each one could affect the TOCs resource plan, which then drives another load of work in rediagramming.

Then, operator B, which runs across abut half the Network of operator A, decides to make a change on August 3rd. All the validation would have to be done again, potentially driving further rediagramming. Then Operator C wants a change on 7th September....

 

[Ianno87]

Accurate modeling of the operations of a timetable would be useful for testing the resilience of the network. You could take an ideal setup, add in some random variations and then simulate it enough times to get good quality statistics.

That is basically done already using simulation software - a package called RailSys is the most commonly used.


Basically, you need a de-conflicted timetable to start off with, then you can apply randomised delays (applied from statistical distributions derived from historic primary delay days) spead over (typically) 100-300 simulated days.

That enables you to pick out trends in a timetable where reactionary delay is most likely to occur (averaged over all days in the timetable), and using manual analysis) relate this back to how this stems from the areas of primary delay.

This is very good at picking out trends in typical days up to a moderate level of perturbation (i.e. most of them). It is less good at severe delay days, as the algorithms that simulate signaller regulation are relatively simplistic, and control interventions like cancelling trains, skipping stops etc cannot be simulated.

Problem is, to do this properly you need to allow time to set up the simulations so they are behaving realistically (months), manually analyse the data and trends to understand what is going on (weeks), be sufficiently in advance of the actual timetable to be able to actually influence it (i.e. by D-55 ideally) and influence operational delivery plans (in the weeks up to the change).


So you basically need to have a concept standard hour timetable (plus peaks) ready to test 2-3 years in advance of the timetable change to give something to tear. And even that will take a good 6 months or so (at least) of thinking about and developing.

But when done, analysed and interpreted correctly, it is a very powerful, and visual, tool to identify the areas where a timetable is likely to perform well, and areas that need focus or refinement.

 

[EllieAquitrain]

That is basically done already using simulation software - a package called RailSys is the most commonly used.


Basically, you need a de-conflicted timetable to start off with, then you can apply randomised delays (applied from statistical distributions derived from historic primary delay days) spead over (typically) 100-300 simulated days.

That enables you to pick out trends in a timetable where reactionary delay is most likely to occur (averaged over all days in the timetable), and using manual analysis) relate this back to how this stems from the areas of primary delay.

This is very good at picking out trends in typical days up to a moderate level of perturbation (i.e. most of them). It is less good at severe delay days, as the algorithms that simulate signaller regulation are relatively simplistic, and control interventions like cancelling trains, skipping stops etc cannot be simulated.

Problem is, to do this properly you need to allow time to set up the simulations so they are behaving realistically (months), manually analyse the data and trends to understand what is going on (weeks), be sufficiently in advance of the actual timetable to be able to actually influence it (i.e. by D-55 ideally) and influence operational delivery plans (in the weeks up to the change).


So you basically need to have a concept standard hour timetable (plus peaks) ready to test 2-3 years in advance of the timetable change to give something to tear. And even that will take a good 6 months or so (at least) of thinking about and developing.

But when done, analysed and interpreted correctly, it is a very powerful, and visual, tool to identify the areas where a timetable is likely to perform well, and areas that need focus or refinement.

Would it be correct to assume that once you have made adjustments based on the simulated results, you then have to put the altered product back through the whole same process all over again?

 

[Ianno87]

Would it be correct to assume that once you have made adjustments based on the simulated results, you then have to put the altered product back through the whole same process all over again?

In the worst case "back to the drawing board" then yes.

If it's more contained (i.e. one particular TOC just wants to do something different without affecting anybody else) then it can be sped up a bit by just re-importing the affected trains.

 

[Ianno87]

I get the impression that a few folk are finding this an interesting read (and I'm enjoying writing it).

If people want to learn more about Operations Planning, I would recommend reading this book: https://www.anharris.co.uk/irop.php

An Introduction to Railway Operations Planning is a translated, expanded and updated version of a book originally written in Norwegian. It starts with a brief summary of how railways work, and the principles of demand forecasting, project development and the information needed for the planning process. The longest chapter of the book is devoted to an explanation of the train planning process, including the preparation of input data from simulations and output data for public timetables. This is followed by specific material on planning other aspects of the operational railway: capacity, personnel, rolling stock, performance, stations and contingencies/special events. Discussion of commercial and energy perspectives is followed by a final chapter on Cost: Benefit analysis and how railway planners can actually get their projects built - and delivering the benefits that were intended.

 

[Paul Kelly]

The other risk is a timetable output that nobody understands. You need to understand where the risks lie in any timetable so that operational delivery teams can be briefed or action plans made (e.g. focusing on dwell times at a key station with dispatch procedures). Human eye picks these things out much better.

That leads to an interesting point. A suitably programmed computer should be able to iteratively optimise a timetable that is easy to understand for humans (consistent platforming, services running at regular intervals and/or minutes past each hour and so on).

I'm not convinced (yet) that this is sufficiently simple or 'black and white' for machine learning to be able to identify a sensible, rational outcome.

I certainly agree. The whole problem is coming up with ways of measuring the timetable to tell the computer how optimal it is.

If you don't mind me saying, it seems to me you may be a little jaded/biased by the shortcomings of existing computer systems that are marketed as solving these problems. I suspect that existing systems focus on too narrow measures of optimalness, e.g. fastest trains, minimum units needed, minimum number of crew needed etc.

For legal track access decision purposes, this becomes a fixed entitity at Version 4 each year - it would leave NR wide open to challenge if it were permanently open to machine-led evolution, e.g. "why did your software improve the rules on favour of that TOCs train, but not my TOCs?" E.g. it would stand open to exposing flaws and holes in the parameters that the system is set to work within.

And that demonstrates that the system exists in part for its own benefit (the cover your backside principle) and not for the benefit of passengers. So that pushes even more in favour of change.

Yes, great point.

And to specify correctly, you need to carefully elicit thousands and thousands and thousands of direct and implicit requirements from a set of the finest timetable planners in the land from Network Rail and the TOCs (who, at the end if the day are timetable planners, generally not computer scientists). And even then you won't get everything.

A more efficient long-term option might be to retrain some train planners as programmers or at least software analysts. I seem to remember reading that something similar was done with the introduction of TOPS.

A university research project would not cut it. You need a large team of experts working full time on nothing else.

Yes I totally agree. People with intimate knowledge of the subject would need to be key members of the team.

if the new magic timetable generator allowed us to update the timetable every 2 months instead of every 6, how would you feel telling passengers in Oxted that their train times in the new timetable had changed beyond recognition to allow a new freight path through the Peak District? And that they would likely change beyond recognition again in another couple of months for a similarly trivial reason?

That wouldn't happen if you were optimising the timetable to be understandable. Avoiding too many confusing changes at once should (in my opinion) be a key part of the rules for constructing it. That allows passengers and crew to get used to the timetable.

Teaching our timetable generator about the quality of a timetable for end users is a whole other level of intractability above just finding a workable solution.

Yes! I think this (bit I've put in bold) is one of the keys to a usable system.

Relentlessly pursuing all the simple automation you can (e.g. checking train diagrams are legal). It sounds like NR have done lots of this.

From what I read in Roger Ford's columns in Modern Railways magazine, this seems to be turning out to be a really useful feature of the new traffic management systems that are being installed in signalling centres.

I guess the simple reason for fixed timetable change dates is the inter-relatedness of the system; call them "opportunities for controlled change";

Interestingly fares is already starting to move away from this model. Although the process of fares only changing 3 times per year (dating back to when paper fares manuals needed to be printed and distributed) is enshrined in many agreements and contracts, more and more TOCs are ignoring this and changing fares randomly all the time, and getting away with it because almost all retail systems are updated nightly these days.

 

[PG]

I get the impression that a few folk are finding this an interesting read (and I'm enjoying writing it).

If people want to learn more about Operations Planning, I would recommend reading this book: https://www.anharris.co.uk/irop.php

Absolutely, this thread is very interesting to study.

Thanks to all who are contributing to the discussion.

 

[Energy]

I don't think anyone is suggesting completely removing train planners, just using computers to do lots of the hard work but humans will still need to check over it to make sure the timetable is ok and give rules to the computer for the timetable wanted. Computers just follow instructions so checks are still needed to make sure another human hasn't made a mistake in the rules put into the computer.

 

[bspahh]

I get the impression that a few folk are finding this an interesting read (and I'm enjoying writing it).

If people want to learn more about Operations Planning, I would recommend reading this book: https://www.anharris.co.uk/irop.php

Thanks. £17 isn't too bad for a new textbook. However, I had a look to see what is available for free.

One of the authors of that book was an author on this article from 2017

NTNU Open: Some Influencing Factors for Passenger Train Punctuality in Sweden

hdl.handle.net

which says "An overview of the state of the art in timetable research is provided by Hansen (2009)."

The Hansen paper is available as a free PDF from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.606.5189&rep=rep1&type=pdf

This is an interesting free PDF review article on "Train Dispatching Management With DataDriven Approaches" from 2019. The authors are from China, but it refers to examples from around the world. http://doi.org/10.1109/ACCESS.2019.2935106

http://doi.org/10.3390/app10020476 is a paper from South Korea, where they predict the dwell time for Metro trains based on the loading of passengers on the train, and the number of passengers who get on and off. At some stations, the dwell time is ~20 seconds, even at the morning peak! It notes:

It is interesting to note that many observed dwell times at certain stations seem to have constant dwell times regardless of passenger counts. This is a result of enforced dwell times at some major stations being assisted by dedicated employees on the platforms, limiting the number of boarding passengers.

 

[306024]

It’s good to see an interesting debate around a subject that few outside the industry could have any detailed knowledge of.

I think we all agree that computers can produce quick and numerous choices. But it can’t be emphasised enough that the humans inputting the rules have to get it 100% right, and the humans analysing the results need time to understand just what the computer has done.

The various crew diagramming optimisation software currently being used require an incredible amount of accurate input to get a decent answer, and still take a while to adjust manually to get the best answer. The inputs for timetabling are a thousand times more complicated than that used in diagramming.

A human building a timetable or diagrams assimilates knowledge in their head as they go. A computer presenting a result takes a lot of time for the human to understand. Also other human factors come in to play, it is easy to become lazy and just accept what the computer has done, especially if time is pressing. Job satisfaction is another issue, there is nothing more satisfying that solving a complex problem. No doubt as computer software develops a different kind of train planner will be required.

Having lived in the era I have, working with everything from sloping tables and graph paper to the latest computer software, I see both sides of the argument. On one level it is a good philosophical debate, but ultimately there needs to be a practical output. A huge human resource with all the detailed knowledge necessary would be needed to set up the National Timetable Planner. In the current climate the industry probably has more pressing issues.

 

[bspahh]

I think we all agree that computers can produce quick and numerous choices. But it can’t be emphasised enough that the humans inputting the rules have to get it 100% right, and the humans analysing the results need time to understand just what the computer has done.

The various crew diagramming optimisation software currently being used require an incredible amount of accurate input to get a decent answer, and still take a while to adjust manually to get the best answer. The inputs for timetabling are a thousand times more complicated than that used in diagramming.

If a system relies on manual data entry, then its always going to be a long slow process to check and validate it. If you are trying to handle a complex dynamic problem, then you need to start by collecting lots of accurate and timely data. That probably means automation. That will cost a lot of money, but it is pocket money compared to the benefits to the economy of an reliable transport network, or the cost of building more infrastructure.

 

[oldchap]

I've found this a fascinating thread. I'm a computer scientist, currently working in a university, and my expertise is in combinatorial and simulation based optimisation. I use stochastic methods (like simulated annealing as mentioned earlier) and exact methods like integer programming, sometimes together (e.g. https://en.wikipedia.org/wiki/Matheuristics). I've worked with a few companies on scheduling, rostering and timetabling problems - including allocating thousands of engineers to shifts and route allocation for aircraft. I have some colleagues who implemented the runway sequencing system for Heathrow Airport. I'm merely an interested observer in railway operations - albeit because it's the kind of complex system that I like to learn about.

Certainly timetabling the whole UK railway system would be a massive job; these kinds of problems, yes, even at this scale, are definitely solvable but there are many challenges! In practice the only way to solve something this big in a feasible time (on any scale of computer hardware) is to compartmentalise the problem into smaller parts that fit together or to start from a working solutions and modify it slightly - both approaches being much the same as a person would do. I've known at least a couple of academics who have worked with Network Rail and TOCs on small parts of the timetabling problem. The biggest challenge, already covered in the thread, is capturing all the constraints of the system, and how to measure solution quality. Much of that is subjective and is a difficult modelling problem in itself, though it has been done in many industries. It can include things like personal preferences; you could imagine a scoring function to measure how close to a clockface timetable each train's times would be, for example, or how many changes might be required for passengers on different routes. Often the process of trying to capture all the characteristics that define the problem ends up being the most useful part of the whole process!

There is a lot of work in optimising multiple conflicting goals (https://en.wikipedia.org/wiki/Multi-objective_optimization) to help identify trade-offs like running cost vs punctuality vs timing simplicity vs ... etc. - these approaches still need human input, they are just about doing the grunt work and helping to make sense of the tradeoffs, so a person can make a more informed choice.

Not sure that's added much to the thread... In short it's definitely a solvable problem for modern algorithms (though it's not a deep learning problem in my view), but probably not all in one glorious whole, and it would need a rather more centralised and automated approach to information gathering to stand a chance of formulating the problem anything like correctly in the first place. The latter part may not be achievable in practice.

 

[bspahh]

Douglas Hartree https://en.wikipedia.org/wiki/Douglas_Hartree built a differential analyser out of Meccano, which he used to calculate timetables for the London, Midland and Scottish Railway. He went on to be a key figure in computational chemistry, with a unit of energy named after him

 

[Worm]

Monte Carlo wouldn’t be a great timetable generator in practice, however there are other options like genetic algorithms or multi-agent systems which will produce a timetable well within the confines of the various minutiae, over many thousands of iterations where each is an improvement of the last.

Although this paper isn’t quite looking at a national paper it describes how a genetic algorithm can be used to optimise a timetable:

https://link.gale.com/apps/doc/A453408713/AONE?u=mmucal5&sid=AONE&xid=ad229bd8

 

[BayPaul]

That is basically done already using simulation software - a package called RailSys is the most commonly used.


Basically, you need a de-conflicted timetable to start off with, then you can apply randomised delays (applied from statistical distributions derived from historic primary delay days) spead over (typically) 100-300 simulated days.

That enables you to pick out trends in a timetable where reactionary delay is most likely to occur (averaged over all days in the timetable), and using manual analysis) relate this back to how this stems from the areas of primary delay.

This is very good at picking out trends in typical days up to a moderate level of perturbation (i.e. most of them). It is less good at severe delay days, as the algorithms that simulate signaller regulation are relatively simplistic, and control interventions like cancelling trains, skipping stops etc cannot be simulated.

Problem is, to do this properly you need to allow time to set up the simulations so they are behaving realistically (months), manually analyse the data and trends to understand what is going on (weeks), be sufficiently in advance of the actual timetable to be able to actually influence it (i.e. by D-55 ideally) and influence operational delivery plans (in the weeks up to the change).


So you basically need to have a concept standard hour timetable (plus peaks) ready to test 2-3 years in advance of the timetable change to give something to tear. And even that will take a good 6 months or so (at least) of thinking about and developing.

But when done, analysed and interpreted correctly, it is a very powerful, and visual, tool to identify the areas where a timetable is likely to perform well, and areas that need focus or refinement.

I'd just like to say that I am finding this discussion fascinating. I work in timetabling for a very different kind of transport (which is also done manually, but with more and more computer modelling to help us optimise), so it is very interesting to see the challenges that you face.

The above quote does sound like an area where better software could do a much better and quicker job of testing the timetable than what you describe. I would have thought a decent semi-permanent model of the way the network works could have been built up fairly simply, that you could 'simply' upload a full national timetable into (and ideally also things like staffing rotas), push go, and have a decent set of answers come out of the end to flag up concerns, and perhaps also suggest potential solutions. The network model could be continually updated with actual performance data, so that it 'knows' that TPE are likely to skip-stop if a train is more than 10 minutes late (for example), and how many people are likely to be on each train, but it could also be 'curated' to manually create realistic days with a range of elements (from speed restrictions and broken down trains, to events causing extra crowding etc), that could be kept in a library to allow every new timetable to be tested against them. Perhaps RailSys already does this, but it doesn't sound ideal - it sounds like a task that could take a few days rather than a few months with a decent tool, leaving more time for the timetablers to do the creative work of building the new timetable.

 

[Ianno87]

I'd just like to say that I am finding this discussion fascinating. I work in timetabling for a very different kind of transport (which is also done manually, but with more and more computer modelling to help us optimise), so it is very interesting to see the challenges that you face.

The above quote does sound like an area where better software could do a much better and quicker job of testing the timetable than what you describe. I would have thought a decent semi-permanent model of the way the network works could have been built up fairly simply, that you could 'simply' upload a full national timetable into (and ideally also things like staffing rotas), push go, and have a decent set of answers come out of the end to flag up concerns, and perhaps also suggest potential solutions.

That is effectively what RailSys can do to some degree (other than the staff and recommending solutions part). Trouble is that big national models take *huge* amounts of computing power, manual update/checking and manual analysis of the "concerns". Just maintaining and keeping up to date such a model is an expensive cottage industry in itself!

The trouble is, of course is that any area of performance challenge is not necessarily just for the timetable to resolve. If anything, changing the timetable should be the last resort.

Take an example like East Croydon or Clapham Junction. You don't need to run a RailSys model to tell you to expect reactionary delay between trains calling one behind the other as a result of dwell time variations. So your options are:

1. "Change the timetable", which at these locations almost always means taking trains out. This then creates a circular problem of putting remaining passengers on fewer trains, and creating dwell time problems either here or somewhere else, or perhaps tightening trains at another junction.
2. Make the current timetable work better: i.e. you use the outputs of RailSys to focus on basic stuff light getting dispatch procedures right, stopping positions in the platform, signallers setting routes promptly, clear plans should trains get delayed in the platform etc, so that there is less variation to dwell times in the first place
3. Infrastructure: even simple things that can be fixed in relatively short timescales, e.g. are the OFF indicators well-positioned, is the TRTS/RA button in the optimal place on the platform.

The first course of action should always be "can the current timetable work better?" rather than "lets mask what is sub-optimal by changing the timetable". This is like comparing apples and bananas; RailSys forms part of, but not all of, determining the actual answer.

The network model could be continually updated with actual performance data, so that it 'knows' that TPE are likely to skip-stop if a train is more than 10 minutes late (for example), and how many people are likely to be on each train, but it could also be 'curated' to manually create realistic days with a range of elements (from speed restrictions and broken down trains, to events causing extra crowding etc), that could be kept in a library to allow every new timetable to be tested against them.

Regulation/recovery strategies and passenger loadings themselves have a circular relationship to the timetable*, as well as being influenced heavily by performance regimes. e.g. delay may be minimised overall by cancelling a train, but the TOC will not do that if that results in a net greater financial penalty, and the relative incentives may vary through the life of the franchise. It's often perverse, but it's reality.


*Good example: Since, May 2018, the 1Txx King's Cross-King's Lynn services have been more commonly known to pick up calls at places like Hitchin during disruption to other services, as there's a bit more natural 'slack' to take them up. Pre-May 2018 you generally wouldn't have done, as it would have messed up the King's Lynn single lines too much.

Perhaps RailSys already does this, but it doesn't sound ideal - it sounds like a task that could take a few days rather than a few months with a decent tool, leaving more time for the timetablers to do the creative work of building the new timetable.

RailSys is itself interesting work when you get to the analysis part; going through data, picking out trends, drawing conclusions, making recommendations. Even sitting with actual signallers playing through example scenarios!