CS789, Spring, 2005

Activity instructions, Week 4

Perform a typical user interface experiment; inspect the results; and submit them to be "analysed".

Experimentation is one of the dominant research paradigms in modern user interface research. In an experiment the researcher tries to find a single element that is common to many user interfaces, and to abstract it into an experimental design where it is the only factor influencing whatever is measured. (As we discussed in the lecture on user models the measured factor is most often time, which is considered to be an indicator of several interesting measures: the time needed to do real tasks in an environment of realistic complexity, the difficulty of the task, and the level of frustration or pleasure experienced by users.)

I have produced an experiment in the form of a Java applet. It can be run by opening the experiment page, but please don't do so until you have read the instructions below.

The experiment task is intended to abstract from the common interface task of filling fields in a form, which consists of two activities, typing the content of the fields and moving from field to field. We want to investigate how best to move from field to field, so the typing part of the task is made as trivial and standard as possible.

We investigate two different methods of moving from field to field: by mouse, and by space bar. To eliminate typing variation we have exactly the same entry to every field, a single carriage return. To standardize geometrical factors we lay the fields out in a standard form that does not vary from trial to trial. You might think that having standardized as much as possible the experiment might be able to conclude that the mouse is better than the space bar, or vice versa. Instead, as is all too often the case we will show that minor changes in the task have significant effects on relative performance. We will consider two task variations, the order in which the fields are accessed, and the use of one or two hands.

With two methods, two orders and two hand configurations eight trials will cover all possible combinations. In the experiment you will do all eight combinations, one after another, and then compare the total time taken to do the entire task.

  1. Start up the applet by going to the experiment page (If you would like to see the source code it is available here. Feel free to copy and modify it if you have access to html display facilities other than the ones I have menitoned in class.) To do the task select the fields in the order written on them, field 0 first, up to field 8. When the field is selected press return, and it will disappear. (The experiment implementation has a known bug. In fact, any key except the space bar works as well as return. I prefer that you use return, but if you must use a different key please use the same key every time.) Then select the next field and so on. Fields are selected by clicking on them with the mouse, or by moving forward a field using the space bar. (The space bar moves down the columns, then from the bottom of one column to the top of the one next to the right. From the rightmost column it moves back to the top of the leftmost column. Completed buttons are skipped over.)
  2. The instruction line at the top tells you how to do the task: mouse or space bar, one or two hands. Take as long as you like figuring out how you wish to do the task; the timing starts only when you complete field zero. When you have done all nine buttons the nine fields are redrawn with a new set of instructions.
  3. When all eight trials are complete the data is displayed in the browser. Transfer it to a mail message and send it to me. Feel free to do the experiment as many times as you like. This will give you a feeling for the variability in your responses.
  4. Can you give an intuitive explanation for the pattern of your results? What bearing might this pattern have on the "experimentally validated fact" that the mouse is a better positioning device?
  5. Do the results of this experiment suggest recommendations on how to implement such an interface? What other things would you like to know, and how might you design experiments that would help you find them out?
  6. This experiment claims to be an abstraction of a field selection task in a form-filling interface. The mouse and keyboard exist as competing position selection methods in many other interfaces, word processors and drawing programs, for example. Would you be willing to draw conclusions about other interfaces form the results you saw in this experiment? If not, how would you design an experiment that would give you results that YOU would trust for some other interface style?

Return toReturn to: