CS789, Spring 2007

Lecture notes, Week 3


One way to regard the interface is as a control problem. The computer receives input from (is controlled by) its input devices, which are controlled by the muscles of the user. It provides output to its output devices. The user receives his or her input from (is controlled by) the (five/six/seven) senses, which are stimulated by the computer's output devices. He or she provides output using muscles, which may or may not be supplemented by actuators. The control problem is dual, a feature that has been little noticed.

Here are several things to think about.

  1. The user is a feedback loop that controls the computer; and the computer is a feedback loop that controls the user. The concepts of control theory apply to these models.
  2. To understand the act of control we need to know:
  3. In general terms:
  4. Two important concepts:

Analogue Control

The simplest example of analogue control involves following a target with a tracker. One target acquisition paradigm -- acquiring an icon with a tracking device -- is the most studied single interface feature in the entire HCI literature: Fitts' law. Using psychological studies of Fitts' law HCI researchers have tried, for example, to link mouse usage to the characteristics of human motor mechanisms. (To study Fitts' law behaviour a target and tracker are simultaneously provided to the user who the 'acquires' the target with the tracker as quickly as possible. Experiments tend to yield expressions of the form
time_to_acquire = A log( D/S + B ) + other contributions
where S is the target size and D is the initial distance from tracker to target. More about this in two weeks.)

Another target acquisition paradigm is the most expensively studied aspect of the human factors literature, the ability of a weapons aimer to maintain his or her sight on a moving target, something of considerable interest -- to understate the case -- to designers of military systems.

We are going to begin by doing a little mathematics, just for illustrative purposes, to show how complicated even very simple problems are. After that there will be a short review of some qualitative features that are important as analogue control is generalized.

What are the dynamics as a user moves a tracker to acquire a target?

We will progressively add complexity.

  1. Stationary target, instantaneous reponse: dx(t)/dt = -a (x(t) - xT).
  2. Stationary target, lagged response: dx(t)/dt = -a (x(t - t0) - xT).
  3. Moving target, instantaneous reponse: dx(t)/dt = -a (x(t) - xT(t)).
  4. Obviously, the real world, even of video games, is much more complex. Fortunately, we have already learned our lesson.

Mathematics of the type written above should immediately make us ask a few questions. For example,

This is a very simple case. When we let things get more complicated qualitatively interesting things start to appear. The following points, which might go under the heading of "qualitative dynamics", are things we think about when thinking about the real-time properties of our interfaces. (If you are interested in doing more than just reading this list please see your nearest applied math textbook on dynamical systems, and work out the analogies to user interface behaviour.)

  1. Dynamical systems are usually defined by the following:
  2. "Degrees of freedom" is an extremely useful concept in analysing dynamical systems:
  3. When control parameters are changed the system normally settles down toward a new state. These states are associated with attractors of the dynamical system. Attractors come in three flavours: Attractors categorize the dynamical space into basins of attraction. Points that separate basins of attraction are places in the dynamical space where infinitesimal changes in the underlying system cause finite changes in the result. These are the places that are both valuable and dangerous. E.g. the set of pixels right that define the border of a pop-up menu. This cannot be adequately stressed; these are the places where quantitative changes in control parameters produce qualitative changes in behaviour of the application. The existence or expectation of hysteresis in the user, or in the system should always be considered near such points.

Discrete Control

Many interfaces depend only on the order in which operations are performed, and not on when they are performed. Furthermore they have particularly simple input/output structures. Consider, for example, an interface that is used for filling and submitting a form, then receiving the results. Draw the state diagram for such an interface:

  1. List of states of program:
    1. Not accepting input.
    2. Accepting input for Field 1.
    3. Accepting input for Field 2.
    4. Processing form.
    5. Form accepted.
    6. Form rejected.
  2. List of states of interface:
    1. No field selected.
    2. Field 1 selected.
    3. Field 2 selected.
    4. Form submitted.
    5. Form returned: correct.
    6. Form returned: error.
  3. Actions that initiate transitions out of states:
    1. New form
    2. Choose field1
    3. Choose field2
    4. Submit form
    5. Remote response to submission
    6. Modal responses to returned forms
  4. Where do the transitions go?
  5. Exactly what action initiates a transition? And who does it, user or application?
  6. Is the interface controllable?
  7. Is the interface observable?
  8. Are there pathologies?
  9. Have you put in something for everything that is "abnormal"?
Doing this provides a pretty good explanation of what is going on. It is, however, extremely tedious.

This technique of interface design has obvious benefits, such as

  1. It's a tool that computer scientists already understand fairly well from other contexts. Transfer of learning.
  2. It's a tool with formal properties that can be exploited. For example,
  3. Application/interface/user synchronization is explicit.
  4. The technique is roughly application-independent.
  5. It can profitably be used for designing parts of an interface. E.g., vi, Adobe Illustrator.
  6. It can profitably be used for explaining parts of an interface. E.g., PBX interfaces.
  7. Automated tools are at least conceivable.
There are also a variety of fairly obvious drawbacks, or at least complications, such as,
  1. There is no way of handling time that is better than temporal order. Time-outs are clearly a hack. (How long do you have to wait at a bank machine before the machine concludes that a transaction is incomplete?)
  2. Complexity kills it pretty fast. Consider the state diagram for an editor.
  3. Obviously more suited to some interface devices and some user tasks than others. For example, producing a business letter might seem similar to filling in a form. How useful is the state diagram we produced above?
  4. How much should a user be aware of the underlying state machine? Is there value to the user independent of such knowledge, which may be implicit?
State diagrams are an obvious place where specific, well-focussed research is possible that will improve user interface design.

History of the State Diagram Idea

Separate the interface from its presentation

  1. Each state has a presentation.
  2. Each input moves the system to a new state.


An interface really is a finite number of states plus transitions between them, BUT

There are two easy to solve limits

  1. number of states -> one
  2. number of states -> infinity

Typical number of states is closer to infinity than it is to one! At infinity we look for algebras as generalizations of state diagrams.


  1. All states accessible.
  2. All transitions inverible.

Look for the closure. What do you get?

  1. Enumerate the states in any order

Return to: