CS452 - Real-Time Programming - Fall 2011

Lecture 14 - Task Structure, Debugging

Public Service Annoucements

1. Due date of kernel 3 (Monday, 17 October)
2. There are two daisy-chained ICUs
3. Kernel 2 results
   • Averages and ranges for each condition
   • Improvement from cache and compiler optimization

Clock Server, Task Structure

A New Kernel Primitive: int AwaitEvent( int EventType )

Argument

1. Somewhere there is a list of event types
   • Application programmer knows the list
   • Kernel can respond to each event type on the list
2. This is not very portable
   • The list would normally be the union of all types occurring on all hardware
   • This is the Windows problem

Strategy for Handling Hardware Interrupts

• Initialization
  • Kernel
    1. Initializes ICU
    2. Creates Idle task, First user task
    3. Goes into request loop and relinquishes CPU to First user task
  • First user task
    1. Creates Server
  • Server
    1. Initializes data structures
    2. Creates Notifier
    3. Sends to Notifier
    4. Goes into request loop and Receives
  • Notifier
    1. Initializes hardware
    2. Turns on interrupt(s) in the device
    3. Receives from Server
    4. Replys to Server
    5. Enables interrupts in the ICU
    6. Goes into request loop and AwaitEvents
• Procedure
  1. Kernel
     1. identifies interrupt source
     2. identifies the correct Notifier
     3. acquires volatile data
     4. puts volatile data in the return value
     5. re-enables interrupt in the CPU during task activation (eg, movs)
  2. Notifier
     1. collects data from kernel
     2. modifies data (if necessary)
     3. Sends data to server
     4. re-enables interrupt in the device (if necessary)
     5. re-enables interrupt in the ICU (if necessary)
     6. AwaitEvents
  3. Server
     1. Receives data from Notifier, possibly eventually
     2. Replys to Notifier
     3. Checks to see if any clients are waiting on the data
     4. Receives

Implementation Decision

1. What should be done if there is an existing interrupt when AwaitEvent is called?

HALT versus an Idle Task

What do you do when there are no tasks to run?

• Idle task
  • lowest priority
  • diagnose system
    • At a minimum it should measure what fraction of execution time is spent in the idle task.
  • search for ETI
• HALT
  • turns off CPU clock
  • save power (battery)
  • provided two ways
    1. through System Controller Co-processor
    2. through the TS-7200 clock controller
  • IRQ path is asynchronous, so it works when the clock is off

You should use an idle task, created during kernel initialization.

• We require you to measure the fraction of time spent in the idle task
• The easiest way is to use the 40 bit cycle counter, which requires the clock to remain turned on at all times.

Clock Server

Primitives

int Time( int tid )

• Clock server starts at zero when it initializes
• Unit of time is tick

int Delay( int ticks, int tid )

• Note error returns
• You might want to add an error for negative arguments
  • ticks is usually calculated, and a negative value is an early warning of falling behind.

int DelayUntil( int ticks, int tid )

• Can be constructed from the above two primitives.

Why is the tid needed as an argument?

There is no swi instruction in these wrapper programs.

Implementation

main( ) {
    notifier = Create( HIGHEST, ... );
    time = 0
    Send( notifier, &evtType, ... );
    FOREVER {
        Receive( &requester, &request, ... );
        switch ( request.type ) {
        case NOTIFIER:
            Reply( notifier, ... )
            time++;
            break;
        case TIME_REQUEST:
            Reply( requester, time,... )
            break;
        case DELAY_REQUEST: 
            Add requester to list of suspended tasks
            break;
        }
        Check list of suspended tasks and reply
    }
}

Comments:

1. You need a common request type, or possibly a union.
2. You should notice a typical server pattern.
   • Notifier updates data
   • Client who can be serviced now is serviced
   • Client who needs service in the future is suspended
   • List of suspended tasks is checked regularly

It's normal to sort the list of suspended tasks. Why?

Debugging Real-time Programs

The most common set of debugging tools used by experienced programmers is the oldest: printf, grep & stack trace.

• The power of these tools is greatly enhanced by strong conventions in code formatting.

Debugging real-time programs, at its base, is just the same as any other debugging, and just the same as empirical science.

1. Gather data.
2. Create a model that explains the data
3. Test the model
4. If the model is not correct, go to 1.
5. Remember that the model is ALWAYS provisional: data collected later may invalidate it, no matter how much data has confirmed it.

But real-time programs are harder to debug. Very few programs are entirely free of critical races, which are the worst type of bug, lurking for weeks months or years in seemingly correct code, the appearing when innocuous, unconnected changes occur.

RedBoot

The memory contents are not wiped by reset. Some of the most difficult errors can be detected only by using the contents of memory after a reset.

On some types of exceptions RedBoot will attempt to connect with gdb. In such cases it writes a bunch of gibberish on the bottom of the monitor screen. Among that gibberish is the address of the instruction that caused the exception. Using the load map generated by the linker you can find

• the function that your application was running when the exception occurred, and
• the assembly language instruction where the exception occurred.

It is usually pretty easy to figure out which line of C source was responsible for the instruction.

Stack Trace

In single-threaded programs this is often the most useful tool.

• Anything that terminates execution abnormally prints the set of active stack frames
• Minimal version
  • name of calling function
  • line number of call
• Extreme version
  • values of arguments
  • values of local variables

What is the equivalent of a stack trace in a real-time multi-tasking environment?

• Most would say it is the set of tasks, and the run state of each.
• That information is available starting at the set of task descriptors, but how do you get it in an understandable way?

Breakpoint

What does it do?

• snapshot of the system
  • This means that computation, including respose to interrupts, must stop, or it isn't a snapshot.
• provides interactive tools for examining kernel data structures, such as
  • task descriptors
  • lists and queues
  • stacks, including the program counter and local variables, of individual tasks
• restart system immediately afterwards
  • If you want to continue where processing stopped you must make certain that all state is saved when you enter Beakpoint and restored when you leave it. What about pending interrupts? You can't stop the entire universe!
  • Otherwise you can re-enter RedBoot.

How do you get it started?

• function call, which you insert in your code when compiling.
  • The easiest and fastest form to implement.
  • Having the call as part of ASSERT is common.
  • Has to exit to RedBoot. (Jump to x00.)
• system call instead of function call, which respects the kernel/user distinction.
• an exception triggered externally
  • at initialization
    1. Set up the system so that the external event will generate an exception
    2. E.g. attach a button to PDIO on the third connector, set up ICU.
  • at run-time
    1. Trigger the interrupt
    2. Switch to Breakpoint in the event handler
    3. Either exit to RedBoot,
    4. Or clean up pending interrupts and resume execution.

Breakpoint is a special case of a particular sort of tool that is very common.

• condition occurs => information is made available
• breakpoint provides the information interactively (`interactively' = `on the time scale of the user')
  • it can stop the system completely. How?
  • but it has limited ability to stop the real world
    • i.e., it hides some bugs

Getting information closer to real-time.

Return to:

• Bill Cowan's Fall 2011 CS452 page
• Bill Cowan's CS452 page
• Bill Cowan's teaching page
• Bill Cowan's home page