CS452 - Real-Time Programming - Winter 2017
Lecture 15 - Serial I/O, Debugging
Pubilc Service Announcements
-
Due date for kernel 4: 17 February, 2017.
-
SRR Performance
-
Model
-
time = t-overhead + t-copying
-
t-overhead = base-oh / (order-oh * cache-oh * opt-oh)
-
t-copying = base-cp / (order-cp * cache-cp * opt-cp)
Base numbers
-
base-oh: 440 usec, 310-700
-
base-cp: 375 usec
Speed-ups: cache - 16(3.1-27.7), order - 1.1(0.7-1.6), opt - 2.6(0.6-6.2)
Serial I/O Timing Inconsistencies
CTS timing.
What you expect to occur when sending to the train controller
-
The train application puts a byte in the hold register. The
byte is moved into the shift register and shifted out.
-
The stop bit is transmitted.
-
A train xmit interrupt occurs.
-
The train application checks CTS.
-
If CTS is set, you put the next byte in the hold register.
Sometimes an implementation based on this expectation surprises
you. In reality what happens when you are sending to the train
controller the following set of events occur.
-
The train application puts a byte in the hold register, which is
loaded and shifted out.
-
The stop bit is transmitted.
-
Two things happen in parallel.
-
In the train controller,
-
The train controller sees the stop bit.
-
Then, probably on the next clock edge, it latches the
character and negates CTS.
-
After one or two clock cycles the byte is clear of the
train UART, and CTS is re-asserted.
-
In the ARM CPU
-
The train xmit interrupt occurs.
-
The train application checks if there another byte to
send, and
-
if there is, it checks the CTS bit in the modem status
register (CTS-sr). What value does it read?
-
When the stop bit was transmitted CTS-sr was asserted.
-
Then, the train controller negates the CTS state
internally, probably on the next edge of its UART
clock; the ARM UART latches the negated signal on the edge of its UART clock,
and into its CTS-sr on the next edge of the peripheral bus clock. From this
point CTS-sr is negated.
-
Finally, when CTS is re-asserted in the train controller UART and has been
latched a second time in the ARM UART CTS-rs is re-asserted.
The train application is assumed to check CTS after the
train controller has negated it. It may even be unaware that
CTS is negated for a short while. Regardless, because the two
sets of events happen in parallel the result is non-deterministic.
-
If CTS is set, the train application puts the next byte in
the hold register.
Whether the final action succeeds, drops a character or
crashes the train controller is determined by the whims of the
gods.
How do you solve this problem is for you to discover. Hint, interrupts
occur in a deterministic order.
Primitives for Serial I/O
We are supposed to support
int Get( int server-tid, int port )
and
int Put( int server-tid, int port, char c )
These are wrappers for sends to one or more serial servers.
-
On Get the server blocks the client until data is available
-
Does a non-blocking TryGet give you anything you cannot do
with task structure?
-
Put is non-blocking. The only guarantee is that the character
has been added to a buffer of characters to be transmitted
some time in the future.
How should we handle a terminal?
Issues
-
Line editing
- undefined intermediate states
- well-defined trigger
-
Echo
Either
-
one server per channel,
-
passing characters from one notifier to the other
Or
- courier connecting receive server to transmit server
- What characteristics does the terminal have? termcap follies
Many other issues come up below as we consider possible task
structures.
How ahould we handle the train controller
Bandwidth of communication with the train controller will probably
be the limiting factor in your trains project.
-
Use it wisely, which means as little as possible.
-
Any time you can substitute computation for communication you should
do so.
Debugging Real-time Programs
The most common set of debugging tools used by experienced
programmers is the oldest: printf, grep & stack trace.
-
The power of these tools is greatly enhanced by strong
conventions in code formatting.
Debugging real-time programs, at its base, is just the same as
any other debugging, just the same as discovery in any empirical
science or any kind of problem solving.
- Gather data.
- Create a model that explains the data
- Test the model
- If the model is not correct, go to 1.
-
Remember that the model is ALWAYS provisional: data collected
later may invalidate it, no matter how much data has confirmed
it.
But real-time programs are harder to debug. Why? There are new
types of bugs.
-
The right thing may happen, but at the wrong time.
-
Small changes in the order of events may cause big changes
in execution. E.g., critical races.
-
The order of events is partly controlled by the real world.
-
The rate at which events occur, and the rate at which humans
perceive, are both very slow compared to the rate at which
program states change.
Critical Races
There is no known method for eliminating critical races.
-
Synchronizing everything, which seems to be an obvious solution,
kills performance because it removes flexibility from the
execution.
It is, in principle, impossible to test away critical races. Why?
- When three trains run continuously for ten minutes, how many events
occur in the real world?
- How many possible orders are there for these events?
- Re-ordering isn't even necessary for a critical race to occur, just
getting too close in time.
Tools you might use to assist debugging
Redboot
The memory contents are not wiped by reset. Some of the most
difficult errors can be detected only by using the contents of
memory after a reset. Produce useful results by inserting
str pc, <magic location>
many places in your code. Redboot can tell you the contents of
the magic places after the program crashes. Then, with the
assistance of a load map, you can find out where you were in which
code when the problem occurred.
You can often find out this information by looking at the garbage
output that often appears at the bottom of the screen. One set
of eight characters is the hex address of the instruction that
could not be executed. With a load map and assembly listings you
can find the instruction on which the program died.
In RedBoot you can, in principle, trace any of the kernel data
structures by examining the kernel's stack, but it's extremely
arduous to do so.
Breakpoint
What does it do?
-
takes a snapshot of the system
-
The body of the program must remain undisturbed.
-
This means that computation, including response to interrupts,
must stop, or it isn't a snapshot.
- It's easy to make a mistake and have some parts of the
program continuing to run while you are interrogating the
system.
-
provides interactive tools for examining kernel data structures,
such as
- task descriptors
- lists and queues
- stacks, including the program counter and local variables, of
individual tasks
- restart system immediately afterwards
- If you want to continue where processing stopped you must make
certain that all state is saved when you enter Beakpoint
and restored when you leave it. What about pending interrupts?
You can't stop the entire universe!
- Otherwise you can re-enter RedBoot.
How do you get it started?
- function call, which you insert in your code when compiling.
- The easiest and fastest form to implement.
- Having the call as part of ASSERT is common.
- Has to exit to RedBoot. Why? (Jump to x00.)
- system call instead of function call, which respects the kernel/user
distinction.
- an exception triggered externally
- at initialization
- Set up the system so that an external event will generate an
interrupt.
-
E.g. attach a button to PDIO on the third connector, set up ICU.
- at run-time
- Trigger the interrupt
- Switch to Breakpoint in the event handler
- Either exit to RedBoot,
- Or clean up pending interrupts and resume execution.
Breakpoint is a special case of a particular sort of tool that is very
common.
- condition occurs => information is made available
- breakpoint provides the information interactively (`interactively' =
`on the time scale of the user')
- it can stop the system completely. How?
- but it has limited ability to stop the real world
Getting information closer to real-time.
Stack Trace
In single-threaded programs this is often the most useful tool.
- Anything that terminates execution abnormally prints the set of active
stack frames
- Minimal version
- name of calling function
- line number of call
- Extreme version
- values of arguments
- values of local variables
What is the equivalent of a stack trace in a real-time multi-tasking
environment?
Return to: