CS789, Spring, 2007
Activity instructions, Week 4
Perform a typical user interface experiment; inspect the results; and
submit them to be "analysed".
Experimentation is one of the dominant research paradigms in modern user
interface research. In an experiment the researcher tries to find a single
element that is common to many user interfaces, and to abstract it into an
experimental design where it is the only factor influencing whatever is
measured. (As we discussed in the lecture on user models the measured factor
is most often time, which is considered to be an indicator of several
interesting measures: the time needed to do real tasks in an environment of
realistic complexity, the difficulty of the task, and the level of
frustration or pleasure experienced by users.)
I have produced an experiment in the form of a Java applet. It can be run
by opening the experiment page, but please don't do so until you have read
the instructions below.
The experiment task is intended to abstract from the common interface task
of filling fields in a form, which consists of two activities, typing the
content of the fields and moving from field to field. We want to investigate
how best to move from field to field, so the typing part of the task is made
as trivial and standard as possible.
We investigate two different methods of moving from field to field: by
mouse, and by space bar. To eliminate typing variation we have exactly the
same entry to every field, a single carriage return. To standardize
geometrical factors we lay the fields out in a standard form that does not
vary from trial to trial. You might think that having standardized as much as
possible the experiment might be able to conclude that the mouse is better
than the space bar, or vice versa. Instead, as is all too often the case we
will show that minor changes in the task have significant effects on relative
performance. We will consider two task variations, the order in which the
fields are accessed, and the use of one or two hands.
With two methods, two orders and two hand configurations eight trials will
cover all possible combinations. In the experiment you will do all eight
combinations, one after another, and then compare the total time taken to do
the entire task.
- Start up the applet by going to the experiment page (If you would like to see
the source code it is available here.
Feel free to copy and modify it if you have access to html display
facilities other than the ones I have menitoned in class.) To do the task
select the fields in the order written on them, field 0 first, up to
field 8. When the field is selected press return, and it will disappear.
(The experiment implementation has a known bug. In fact, any key except
the space bar works as well as return. I prefer that you use return, but
if you must use a different key please use the same key every time.) Then
select the next field and so on. Fields are selected by clicking on them
with the mouse, or by moving forward a field using the space bar. (The
space bar moves down the columns, then from the bottom of one column to
the top of the one next to the right. From the rightmost column it moves
back to the top of the leftmost column. Completed buttons are skipped
- The instruction line at the top tells you how to do the task: mouse or
space bar, one or two hands. Take as long as you like figuring out how
you wish to do the task; the timing starts only when you complete field
zero. When you have done all nine buttons the nine fields are redrawn
with a new set of instructions.
- When all eight trials are complete the data is displayed in the
browser. Transfer it to a mail message and send it to me. Feel free to do
the experiment as many times as you like. This will give you a feeling
for the variability in your responses.
- Can you give an intuitive explanation for the pattern of your results?
What bearing might this pattern have on the "experimentally validated
fact" that the mouse is a better positioning device?
- Do the results of this experiment suggest recommendations on how to
implement such an interface? What other things would you like to know,
and how might you design experiments that would help you find them
- This experiment claims to be an abstraction of a field selection task
in a form-filling interface. The mouse and keyboard exist as competing
position selection methods in many other interfaces, word processors and
drawing programs, for example. Would you be willing to draw conclusions
about other interfaces form the results you saw in this experiment? If
not, how would you design an experiment that would give you results that
YOU would trust for some other interface style?
Return toReturn to: