Lecture 24 - Multiple Trains

Public Service Annoucements

1. Train Control I demo on Friday, 11 March.
2. The exam will start at 12.30, April 19, 2016 and finish at 15.00, 20 April 2016.
3. Suppose you decided in advance that you want a train to travel at exactly 28 cm/sec.
• It is very unlikely that there is a speed at which the train travels that velocity.
• It is almost certain that there is a closest speed above 28 cm/sec, Sa, and a closest speed below, Sb.
• Obviously you want the train to travel at Sa some of the time and at Sb some of the time. How much of each?
• If the velocity you want is, V, and the calibrated velocities for Sa and Sb are Va and Vb respectively, you might try
(1 - x)*Sa + x*Sb with x = (V - Va) / (Vb - Va).
• Naturally, this will require a little tuning.
• You might try something like the short moves method.
• You might experiment with the granularity of changing from one speed to another.
• Dynamic tuning is also possible.
• Each time you hit a sensor you get a measurement of V.
• From that you can calculate a new value of x, x-new.
• Then update x by x = (1 - a)*x + a*x-new.
A similar approach can be used to have one train follow another at a specific distance.
• There is a relationship between the time required for a sensor poll and the minimum distance at which the leading and following trains never collide.

Multi-Train Control

By the next milestone you will be able to control two trains at the same time.

To accomplish the first milestone you sorted sensor responses from the train controller into two categories:

1. ones caused by a train, and
2. ones you ignored.
For the next milestone you split the first case into two: ones caused by train 1, and ones caused by train 2. In other words, Which train triggered which sensor?
• As long as the trains are sufficiently far apart this is not too hard.
• What is the meaning of `sufficiently' in practice?
• Sensor attribution must function correctly in the face of single failures,
• of sensors, or
• of turn-outs.

One way of doing this is to plan ahead.

• I expect the train to pass sensor X at time T, with a margin for error of 2\DeltaT.
• T comes from your velocity calibration: T = X/v
• \DeltaT also comes from the noise in my velocity calibration, and may not be symmetrical around T.
• That means that I expect the sensor
• Not to have been triggered before T - \DeltaT
• To have been triggered before T + \DeltaT
• For each of these normal things you should have things ready to do if they don't occur.

Communication bandwidth to train controller

This is the scarcest resource.

The symptom that you are trying to use it too much is getting a lot of time-outs for events that actually occurred. Switches switching too late is another symptom.

• The usual cause is buffers filling in a serial server.
• If your programme is requesting sensor reports faster that the train controller can provide them then the time between asking for a report and getting the corresponding report lengthens monotonically. Growing buffers and queues in servers is an early warning of performance problems.

Route Finding and Following

You need to be able to route in the presence of obstacles. Some obstacles are stationary; others are moving: you should handle both properly. It's normally a good idea to start with stationary obstacles. Blocking one or more sections of track by a keyboard command, then asking for a route is probably the easiest place to start.

You need routes that reverse because they improve the performance of your project.

• As you add trains to your project, each train sees more blocked track when it asks for a route.
• The demo hits a limit when no train can find a route to its destination.
• Thus, more and shorter routes means more success in keeping your trains moving.
Please remember that it's not enough to find a route; you must also be able to drive a train along it. Driving over a route to a destination is not too hard, but it must be very robust because it's a basic capability required for driving two trains at once.

You can try a gradually harder approach.

1. Make sure that you really have the train finding the shortest route using only one train.
2. Make sure that you can route around one or more obstacles by manually removing an edge from the graph. You should be able to do this interactively at the keyboard, and you probably want to display on the terminal which edge(s) have been removed from the graph.
3. Drive a second train to some point; make sure that you route around it automatically.
4. Let the second train move in a simple way; make sure that you can route around.
5. Make two trains route simultaneously.
When you are testing, calculate mentally, or by hand, the shortest route before you start the train moving.

It doesn't matter what shortest-path algorithm you use, with one exception: the Floyd-Warshall algorithm does NOT work.

When calculating the length of a path you might want to add some extra distance every time the train has to turn around. That is, you are probably most interested in the time a train takes getting to its destination, and reversing adds significant time.

Driving a Train

Driving a train along a route seems easy. But maybe I am wrong to think so because many students seem to have trouble making it robust. It's highly desirable to have route following robust enough that it's not giving you trouble. Here's my suggestion, based on starting in a state that's easy, then making it harder.

1. The easiest way to follow a route is to put all the switches into the correct state, then drive the train along the route at a modest speed, maybe 6, at which the train moves reliably and your software has no trouble keeping up with it.
2. The speed up making certain that you don't fall behind in knowing where the train is.
3. In your project several trains will be using the track at once, and you can't assume that other trains won't want to use some of your route before you get to it. To minimize interference you must switch as you go. (We will see later that coordinating switching and reserving track is very natural.) Thus, you need to follow the route switching switches as you come to them. Make the switches wrong before you test to ensure that you are not just being lucky.

Collision Avoidance

This would not be too hard if the trains stopped instantaneously, but they don't.

• You must look ahead into the unknown, and to some extent unknowable, future.

Your train program must plan ahead, far enough ahead that two trains on a collision course will be stopped before they collide. That means you must modify the routes and/or speeds of trains when they are as much as two stopping distances apart: about a metre.

It is usually your method of collision avoidance that limits the number of trains that can run simultaneously.

• N trains start running.
• Then, after a while they get frozen and won't move.
• Each is waiting for another to move.
• If N is large freezing usually occurs because the route finder can't find a route for any train.

I like distributed solutions, where each train operates -- plans, drives, make decisions, etc -- as though there are no other trains on the track. Why do I like this?

• Global optimization algorithms are rarely (=never) real-time.
• The code is simple.
• Each task has a well-defined role. E.g.
• Track monitor knows the current state of the track.
• Route planner uses track monitor input to provide a feasible route.
• Train driver gets a route and follows it switching turn-outs ahead of itself as it drives.
• Structure of inter-task communication arises naturally from the roles.
• The solutions usually scale well, requiring only that the track length grows at least linearly with the number of trains, which it doesn't in the trains lab. That's why there are limits, though we don't know them.

Treating the track as a shared resource

Analogy to pixels in a window environment.

A track server gives out and takes back pieces of track.

What policy should it have?

1. Trains can only occupy track they have obtained from the server.
2. The server never gives out pices of track that are already out.
3. To avoid leapfrog deadlock, all the track owned by a train must be contiguous.
4. Track should be returned to the track server as soon as a train leaves it.

The first two are necessary for correctness. Are they sufficient? The third avoids a common bug. The fourth both avoids bugs and improves performance.

Reservations

Somebody has been doing something right for over a century. The answer is reservations.

Two Level Train Control

The two levels are completely independent of one another.

• On heavily used sections of track the lower level is done completely by hardware with no possibility (almost) of human intervention

Upper Level

1. Train asks dispatcher for a route
2. Dispatcher provides a route that he/she thinks to be conflict free
3. Train follows the route, reporting back to the dispatcher as landmarks (sensors) are passed.
• The dispatcher gets two reports
1. One from the hardware
2. One from the engineer
• It is up to the dispatcher to make certain that they do not conflict.
• What is to come on the route is communicated to the train driver by the coloured lights you see along the track

Lower Level

The lower level is also communicated by the coloured lights. In cases of conflict between the upper and lower levels, the lower level overrides the upper level.

• Everything is rigidly enforced by hardware
• The human enters the loop only in that the lights tell the engineer what he/she is allowed to do
• The engineer loses his licence, FOREVER, if he/she ever goes through a red light.
• If the system detects a violation of its rules or a state that should never occur it enters a failsafe mode: all lights red.

Something Essential that You Must Do

Design your reservation system before coding it.

Before coding your reservation system work it out on paper and make sure that it works for all the generic cases you can think of

1. One train following another
2. Two trains on a collision course
3. Short routes with much switching
4. Single point failures.

There are one or more switches in the path

The Magellan Spacecraft

Error Handling

We have mentioned several times in this course that when you detect an error

• usually by seeing something go wrong on the track or the terminal,
• but there are other ways,
you have three choices,
1. adjust internal parameters and see if the error persists,
2. find the error and correct it using the usual editor/compiler method, or
3. add code that repairs the error on the fly.
The third option sounds hard, but there are specific cases in which you must repair the error on the fly. The Magellan spacecraft is an interesting example.

The Magellan Spacecraft

Its task was to fly to Venus, get into a stable orbit around Venus, and fly around Venus in a helical orbit looking down with its radar, mapping the distance from it to whatever is solid enough to reflect radar. That would give us the first glimpse of what Venus looks like below its poisonous clouds. After several years of doing this, day in, day out, it was to sacrifice itself in the cause of science, diving straight to the ground, measuring the chemistry of Venus's atmosphere and radioing back data until it could do so no more.

On its way to Venus the spacecraft had to navigate itself, just like our trains, figuring out exactly where it was from small amounts of data. It did so by pointing itself directly at the sun so that its solar panels were perpendicular to the sun, which fixed one direction. It also had a small telescope and it kept a star right in the middle of the field, fixing its other two directions. With this knowledge it was able to point a highly directional antenna directly at the Deep Space antenna on earth, enabling communication.

Consider a spacecraft, millions of kilometers away. You can't press the reset button. Here's how the Magellan spacecraft handled such a problem. Start with the symtoms.

1. The spacecraft arrived at Venus travelling too fast to be captured by Venus's gravity. At a critical time it had to fire its thrusters to transfer itself from an orbit around the Sun to an orbet around Venus. The firing occurred when the spacecraft was behind Venus, out of communication with Earth.
2. The spacecraft passed behind Veus and lost communication with teh ground station.
3. The engineers on earth waited for it to come out from behind Venus and resume comunicating. But it didn't do so.
4. The earth antenna started broadcasting a guide signal toward Venus.
5. Twenty-one hours later, the spacecraft said hello, by sending a prompt, '>'.
6. The earth antenna sent the four bytes CRUZ followed by an operating system kernel, which the spacecraft installed, like RedBoot.
7. Five days later they again tried to change the orbit.
8. This time it was only thirteen hours before the spacecraft answered.
9. By this time the problem had been determined by earth-bound engineers, and a bug-free kernel was sent to it. Magellan went on and completed its entire mission without further problems.

1. Periodically the small telescope looked at the guide star and the computer fired its thrusters to correct the spacecrafts attitude.
2. On this occasion an image of the sun, reflected into the telescope at an oblique angle, then reflected again from the inside of the telescope barrel and onto the photodetector array.
3. The reflection was the brightest thing in the sky and the computer thinking it was the guide star fired the thrusters.
4. The computer, getting impossible valuesas input, wrote over its instruction memory and crashed hard.
5. The watchdog timer timed out, and the computer rebooted.
6. At this point the spacecraft was tumbling ened over end. As part of rebooting the computer had to find the direction to earth to get help.
7. Adding a helical scan to the tumbling allows pointed the solar panels in every direction until it found the sun: the first axis was established.
8. Small antenna now scanned in every direction until it found the guide signal from earth; communication was re-estabished.