CS452 - Real-Time Programming - Fall 2011

Lecture 19 - Servers, Courier, Warehouse

Public Service Annoucements

1. Due date of kernel 4 (Friday, 28 October)
2. Saturday November 5, 11.00 to 13.00. Open house for high school students.
3. Information session on graduate studies: Tuesday, 8 November, 2011 at 16.30 in MC 2065
4. Final exam: Friday, 9 December, 2011 at 09.30.

Anthropomorphic Programming

1. Proprietor with a Notifier

2. Using a Courier

Notes

This gets you through a bottleneck where no more than two events come too fast.

Remember that all the calls provide error returns. You can/should use them for error recovery

• static error recovery: debugging
• dynamic error recovery: at run time

Another possible arrangement for task creation

• Server creates the courier
• Couier creates the notifier

Another possible arrangement for initialization

• Server Receives
• Courier sends to its parentTid
• Notifier sends to its parentTid

Distributed gating

I am showing you collections of tasks implemented together because sets of related tasks is a level of organization above the individual task.

E.g., the decision to add a courier requires revision of code within the group, but not outside it.

3. Using a Warehouse

Note

This structure clears up most problems when a burst of requests to the server would leave the notifier waiting in a long sendQ.

• Warehouse and proprietor share the work.
  • Server's Tid is public; Warehouse's Tid is private.
• This is far from the only way to share the work. For example,
  • The server could be guarded by a receptionist (see below) who ensures that another client request occurs only when the previous request is complete. Then the warehouse is unnecessary.

Two issues:

1. Handles bottlenecks of all sizes.

   Give a precise and quantitative definition of `bottleneck'.

2. Server could be buffered on the other side

   Called a receptionist, sometimes `guard'..

What this amounts to is

Server should be lean and hungry

• do as little as possible

always be receive blocked

4. The Receptionist

We can do to the clients what the warehouse does to Notifiers.

• The receptionist can discriminate amount clients, who may differ in priority
• Receptionist's tid is public; proprietor's tid is private.

Implementation

Receptionist

Working

• FOREVER
      Receive( request )
      switch( request.type ) {
      case CLIENT:
          if( prop ) Reply( prop, req )
          else enqueue( reqQ, request )
      case PROPRIETOR:
          Reply( client.tid, client.result ) // Result sent by proprietor in received request.
          if( ! reqQ-empty ) Reply( prop.tid, dequeue( reqQ ) )
          else prop = request.tid

Proprietor

Now should be called the President!

Doesn't actually do much work at all any more.

• Could be as simple as a courier

  FOREVER
      Send( receptionist, result, request )
      Send( warehouse, request, result )

  In this case the warehouse doesn't actually need a courier

• But the there's no reason to change the proprietor,
  • just interpose a courier
• With more than one courier,
  • the receptionist can prioritize
• Of course, the receptionist could do all the prioritizing by itself,
  • by keeping several queues

5. Administrator, Worker

Administrator is a proprietor who does no work but only assigns work to others

• Tasks are given to workers
• If Create is fast and you have a method for reclaiming resources you can Create and Destroy workers on demand.
• Otherwise workers are created at initialization and the administrator maintains a pool of free workers
• each free worker is a REPLY-BLOCKED task

Real administrators manage workers

• Static ordanizations hire a workforce of employees who are assigned tasks as they come up.
• Dynamic organizations hire workers after the need for work appears, and lay them off when the work is done. These are called contract workers or consultants.

Most workers prefer employee status to consultant status, and so should you.

Worker code

  Send( administrator, nil, workOrder );
  FOREVER {
    result = work( workOrder );
    Send( administrator, result, workOrder );
  }

Administrator code

Initialization

Administrator( ) {
  for ( i = 0; i < NUM_WORKERS; i++ ) worker[i] = Create( workerCode );
  for ( i = 0; i < NUM_WORKERS; i++ ) {
    Receive( *worker[i].tid, ... );
    enqueue( employeeQ, worker[i1 );
  }
    
  FOREVER {
    Receive( requester, request/result );
    switch( request.type ) {
    case CLIENT:
      enqueue( orderQ, {client = requester, request} );
      if ( !empty( employeeQ ) ) Reply( dequeue( employeeQ ) ), dequeue( orderQ ) ); 
      break;
    case WORKER:
      Reply( result.client, result );
      enqueue( employeeQ, requester );
      if ( ! empty( orderQ ) ) Reply( dequeue( employeeQ ) ), dequeue( orderQ ) ); 
      break;
  }

Note: Tid of the client is included in the workorder to the administrator does not have to maintain and search a list of jobs being done. (Administrators are by nature lazy!)

Alternative Worker/Administrator Model

1. As above, Administrator includes Tid of the client in the order.
2. Worker replies to client with result and to administrator with request for another order to process.

Comments

1. The administrator can add a little more value.
   • Suppose that there is data required for processing each order.
   • The administrator could receive it from a notifier or courier.
   • It would be maintained internally and added to the appropriate order before the order is despatched to the worker.
   • Another model: the worker queries a detective.

6. The Detective

Simple Events

The notifier is a task that waits on events.

• AwaitEvent is like Send

  It has two features

  • a simple, kernel-defined event that it waits on
  • hardware/kernel is like Receive/Reply
  • can only serve one master
• The notifier needs to pass on that the event has happened
  • which it does using Send

    FOREVER {
      AwaitEvent( eventId );
      Send( server );
    }

You could call a notifier a detective,

• who looks around on your behalf,
• and let's you know when something you care about happens,
• but really it is a detective's worker.

Complex Events

In an application there is likely to be lots of waiting on combinations of events.

• form the combinations using Boolean operators
• for example, Respond `okay' when this sensor is triggered or `time-out' when Delay returns.

We use the detective to discover that a complex event has occurred.

• How does the detective work?

Conjunction

Code could be

FOREVER {
  Send( part1 );
  Send( part2 );
  ...
  Send( master );
}

Disjunction

Code above doesn't even pretend to work!

Try instead

FOREVER {
  Receive( *requester, request );
  switch( request.type ) {
  case CLIENT:
    if ( request-satisfied( dbase ) ) Reply( request.tid,  )
    else enqueue( requestQ, request )
  case WORKER:
    enqueue( employeeQ, requester );
    update ( dbase, request );
    foreach request in requestQ
      if ( request-satisfied ) Reply( request.client, ... );
    break;
}

This is the code of a detective.

Not

We can say that an event has not happened yet.

Only at the end of the universe can we say that an event has simply not happened.

Time-outs are needed for NOT

• OR with Delay on the clock server.

Who is the client of the detective

1. Initiator of a normal action
2. Housekeeper of the system who will clean up pathologies (Idletask?)

Debugging Real-time Programs

The most common set of debugging tools used by experienced programmers is the oldest: printf, grep & stack trace.

• The power of these tools is greatly enhanced by strong conventions in code formatting.

Debugging real-time programs, at its base, is just the same as any other debugging, and just the same as empirical science.

1. Gather data.
2. Create a model that explains the data
3. Test the model
4. If the model is not correct, go to 1.
5. Remember that the model is ALWAYS provisional: data collected later may invalidate it, no matter how much data has confirmed it.

But real-time programs are harder to debug. Very few programs are entirely free of critical races, which are the worst type of bug, lurking for weeks months or years in seemingly correct code, the appearing when innocuous, unconnected changes occur.

RedBoot

Stack Trace

Breakpoint

What does it do?

• snapshot of the system
  • This means that computation, including respose to interrupts, must stop, or it isn't a snapshot.
• provides interactive tools for examining kernel data structures, such as
  • task descriptors
  • lists and queues
  • stacks, including the program counter and local variables, of individual tasks
• restart system immediately afterwards
  • If you want to continue where processing stopped you must make certain that all state is saved when you enter Beakpoint and restored when you leave it. What about pending interrupts? You can't stop the entire universe!
    • Saved state makes it possible to show the user the contents of the registers.
  • Otherwise you can re-enter RedBoot.

How do you get it started?

• function call, which you insert in your code when compiling.
  • The easiest and fastest form to implement.
  • Having the call as part of ASSERT is common.
  • Normally exits to RedBoot.
    • Try something like

          mov   pc, #0x20

    • What happens if the function simply returns?
• system call instead of function call,
  • which respects the kernel/user distinction.
• an exception triggered externally
  • at initialization
    1. Set up the system so that the external event will generate an exception
    2. E.g. attach a button to PDIO on the third connector, set up the ICU to enable parallel DIO interrupts.
  • at run-time
    1. Trigger the interrupt
    2. Switch to Breakpoint once you get into the kernel
    3. Either exit to RedBoot,
    4. Or clean up pending interrupts and resume execution.

Breakpoint is a special case of a particular sort of tool that is very common.

• condition occurs => information is made available
• breakpoint provides the information interactively (`interactively' = `on the time scale of the user')
  • it can stop the system completely. How?
  • but it has limited ability to stop the real world
    • i.e., it hides some bugs

We need methods of getting information closer to real-time

Return to:

• Bill Cowan's Fall 2011 CS452 page
• Bill Cowan's CS452 page
• Bill Cowan's teaching page
• Bill Cowan's home page