CS789, Spring 2007

Lecture notes, Week 6


Synchronization is about time: when should something happen? Sometimes real-time matters (double clicking); sometimes only temporal sequence matters (typing a command to Unix). Synchronization is also about context: in what context are the actions of the user interpreted by the computer (commands vs input typed to vi), and vice versa (<P> in html source versus in formatted html)?

Why are context and time related like this? Because human ability to handle different contexts simultaneously is limited. Often, when an interface offers several contexts simultaneously the user operates within only one at a time.

As an example, I can imitate your facial expressions. How?

So, why can't two people having a conversation talk at once? It should be possible. What do I have to do in the second case that I don't have to do in the first one? (Hint. Most of us can read from a tele-prompter, but very few can read and paraphrase at the same time. What does simultaneous translation tell us?)

A simple example: the prompt

Most interfaces exchange control between the user and computer. How is the exchange initiated and completed? The key concepts are the trigger and the prompt.

What makes a good trigger?

A few examples of triggers:

What makes a good prompt?

A few examples of prompts:

Sequential synchronization (synchronization protocols)

The simplest forms of synchronization exist to ensure that things get done in the right order. Let's look at three forms of synchronization, taken from the virtual input device models of the GKS graphics standard.

There are three conceptual modes of human/system synchronization

  1. Request mode, where the system requests input from the user. E.g., Unix. The process is explicitly sequential.
  2. Event mode, where the user provides the system with things to which it must respond. E.g., typing to a word processor. The user doesn't have to wait for the processing of a previous event to finish before sending a new one. Maintaining and showing the context in which the new event will be interpreted is not necessarily simple.
  3. Sample mode, where the system tries to follow the user. E.g., free-hand drawing with controls points appearing, as in Illustrator. Time is normally an essential part of the system's interpretation of the input.

Note that while the trigger is a useful concept in the first two cases it's less useful in the third. We can usefully make the following distinction between two input styles:

We may conceptualize synchronization in terms of handshaking, but it is important to note that the prompt concept actually elides three separate actions that users may wish to distinguish. Separating them out an interface transaction can follow the sequence:

  1. prompt -- the system can accept new input,
  2. trigger -- input is ready to be accepted,
  3. echo -- here is the input that was accepted,
  4. acknowledge -- input has been processed.

In Unix the four stages are easily distinguishable in the Cshell when the command !! is issued.

% !!                                # prompt and trigger
/usr/sbin/shutdown -y -i5        # echo
<blank screen>                        # acknowledgement

One reason for differentiating the three components is that we might want to reorder them. Why? (Hint. "&" in Unix, which gives a new prompt before acknowledge.)

Now, what about context?

The second important aspect of synchronization has to do with the coexistence of content and context, their interrelationship in creating meaning, and the necessity of keeping them together.

When the system provides output to the user,

When the system takes input from the user,

Slippage between context and content, (The ATM thinks I'm putting in the amount of money I want; I think I'm putting in my PIN number.)

gives either user or system incorrect information. How is slippage best avoided? Give some examples to show that the problem is real.

Solutions depend on continuity, which is takes advantage of the user's memory. How is continuity broken? The user's continuity is broken by interruptions (which are, by definition, unexpected),

which cause a change of context in the user. Note that in some cases the user can change context as slowly as he or she likes, leaving ample time to arrange context-retrieval; in other cases the response has a time limit, limiting the opportunity for arranging context-retrieval. How can the user regain the right context after an interruption? The system's continuity is broken when changes occur in the system. Such changes must be drawn to the attention of the user. There are two strategies for doing so. As an example, consider strategies for managing focus in window systems.

Multi-stream synchronization (windowing systems)

The problem of multiple contexts becomes significant when the human computer interaction is spread over several tasks and interfaces, as is typical in modern windowing systems. Error free synchronization of system resources, screen, mouse, keyboard, memory, CPU, and so on, is hard enough; synchronizing the user is harder still.

Real-time Interfaces (multi-media)

Real-time has always been with us: humans operate in real-time. Time-outs must exist in any interface, but only as a violation of sequential semantics!

Multi-media concepts

Modality What is the difference between spatial and temporal modalities? Each perceives objects consisting of parts. In a spatial modality the parts can be visited in any order; in a temporal modality the order is fixed by our one-way passage through time.


Streams are

Obviously modality and medium, while useful as general terms, are not very precise. They are useful when attaching characteristics to capabilities: CRTs have different output characteristics than LCDs; eyes have different input characteristics than ears. On the other hand they are less useful when we try to describe interface architecture. For this purpose stream is a much more useful basic concept.

Using the above terminology it is obvious that windowing systems are multi-stream systems. They usually have a single output medium, a CRT, connected to a single modality, vision, and making available several output streams (windows), interacting with a single attentive (human-)input stream that is time-multiplexed between windows. They usually have two (computer-)input streams, mouse and keyboard, both associated with the same user modality, hand/finger. Control of the (human-)output modality is time-multiplexed over the two input streams.

Multi-media systems, as the name suggests, are systems that employ more than one medium for communication with the user. From the user's point of view it is most significant that different streams contain correlated input. Users seek out correlation among the many input streams they process. Correlation is based on simultaneity.


Multi-media architecture

The most powerful new interface technique available in multi-media systems is the ability to have more than one stream of information from a single conceptual object. To provide this it is necessary for a system to provide application support for simultaneity with a granularity close to 10 milliseconds, and to maintain this synchronization over long periods of time. (Streams broken apart are not easily put back together.) We are still waiting for general solutions to this problem.

Return to: