CS789, Spring 2005, Lecture notes, Week 10
CS789, Spring 2007
Lecture notes, Week 10
Why do they exist?
How do they function?
- learning transfer by way of
- story telling
Fundamental to user experience
- Newton's law of inertia, with friction
- things stay where you leave them
- things don't change without a cause
- Conclusion, changes of position are significant.
Icons and Direct Manipulation
Separate dialogue styles exist along a continuum:
linguistic-->graphical, menus and forms in the middle. Linguistic is based
on words, talking:
Graphical is based on symbols, looking:
- auditory, serial, temporal
- "You can't talk and listen at the same time."
- alternates listening and talking in significant chunks
- compare feedback in OS interaction to conversational feedback
E.g. contrast "rm myfile", with moving myfile icon to the trash, with
selecting icon and choosing "delete" from a menu.
- visual, parallel, spatial
- "You can see (active), monitor (passive), and act at the same time."
(There are even background actions.)
- continuous monitoring of the activity
- metaphor includes something that moves continuously toward the goal
This is a contrast between
- In each case an action triggers the disposal of the file: carriage
return at the end of a command line, releasing the mouse button with item
over the high-lighted trash, releasing the mouse button with "delete"
- In each case there is real progress toward a goal,
- but the feedback is very different: lengthening string of characters,
increasing nearness of icon to trash, progress in selecting icon &
menu command activation.
- commands with names as objects
- commands with icons as objects
- icons as commands and as objects.
General organization of graphical dialogue styles:
- Graphical symbols (icons) designate objects. Symbols have a spatial
location, on the display surface.
- Important distinction is discrete versus continuous (nominal/ordinal
- icons: static graphical representation, discrete set of underlying
- position: continuous
- graphical objects: variable graphical representation, continuous
set of underlying objects
- Let us distinguish direct manipulation from graphical interfaces.
- direct manipulation
- icons, each of which has a position, and a reference
- icons are unchanging; icon characteristics are only a
convenience for the user, helping the user make the correct
reference. For example
- On a bank machine, instead of "WITHDRAW CASH",
- we might have a pile of cash inside an account icon,
- we drag the cash to wallet icon.
- Colour of the wallet probably isn't significant.
- What about the colour of the bills?
- locations of icons change; relative location indicates
operations to be performed
- graphical interfaces
- graphical objects, each of which has a position, and a
lot of other properties
- the other properties of graphical objects can be manipulated,
most often as they are because the screen object is actually a
picture of the real object
- locations of objects also matters, and makes possible a complex
- We need to define and exploit possible interface operations. For
The problem here is that there are insufficient operations to exploit:
the mouse buttons have insufficient bandwidth to occupy our full
attention. We want the amount of activity generated by our motor
systems to match the bandwidth of our perceptual systems to monitor it,
and the bandwidth of our cognitive systems to keep track of it.
- empty space: create
- on an icon: select
- on part of a graphical object: select
- empty space: ??
- on an icon: open/execute
- on part of a graphical object: expand (show sub-parts)
- starting in empty space: area for multiple selection
- starting on an icon: move the icon, end-point dependent
- starting on part of a graphical object: change parameter(s) of
- Here are three ideas for increasing the number of things we can do
Have you seen any of these interaction techniques? Why do you think they
are used so little?
- shift-point-and-click, control-point-and-click, etc.
- point-and-click-B1-then-B2, point-and-click-B3, etc.
- palette with different cursors that change the meaning of
In the end these interaction techniques show
- current limited use of direct manipulation techniques
- substantial promise of direct manipulation techniques
- chicken and egg problem analogous to how you get from bacterial purple
to the eye.
What is an icon?
- graphical name
- names an object known to the interface
- its internal structure is not directly accessible, but is often
indirectly accessible through a form
- identity, location
- subject to a limited set of operations
Why use icons instead of words?
- help the user to remember: icons can even be animated. This points out
the interesting double existence of icons.
- visual distinctiveness: a richer set of perceptions related to finding,
What factors make icons visually distinctive? Distinctiveness is a
property of a set of icons!
- some aspects of shape: angles, lines, curves
- usually not conjunction or disjunction
- Select one of several distinctive dimensions.
- Processing more than one dimension is hard.
- your eye will make a good decision
- you can always do simple tests
- Can you see the pattern?
- Is search time independent of distractors?
but it's likely that you care about peripheral distinctions, not
How are selection actions carried out by the user?
- click in the space occupied by the icon: "Click on the icon."
- multiple select one
- multiple: indicate space containing several icons
- Draw out the space
- Text is a linear space: indicate beginning and end.
- Take advantage of granularity: spreadsheet, text
- Are there useful granularities for other objects in the interface?
rows, columns, clusters, everything?
- expand or contract
It's interesting to note that discontinuous selection, which seems to be
a hard interface to produce for linear structures, is a fact of life for
unstructured collections. Why?
- toggle, as in shift-click
- new end-point, for linear structure
- differentiated selection
- multiple, differentiated arguments
- feedback is what's hard, showing the user the current state of
Why do users change the placement of icons?
How do users change the placement of icons?
- for multiple selection
- for visual clarity
Two methods for supporting grouping of icons
- two arguments: what to place, where to place it
- what by selection
- where by selection
- how prominent should the path be?
- containment, files directories - relies on memory
- icons with common properties - relies on perception
How do we indicate commands to be carried out?
There are some unexploited possibilities in current interfaces
- Select object of command; give command
- selection indicates the arguments
- How are commands given?
- mouse action, such as double-click
- hot key on keyboard
- special destination of a drag (move command)
- differentiated multiple selection for multiple arguments: exchange
selection 1 and selection 2
- icon identity: delete all objects with icons like the selected one
- drag path: print then delete
Differentiate interface operations (with pointer) from the actions they
entail (create). Typical operations
Note that there aren't very many of them. How do we make more?
- position: where or what
- click: trigger
- down-drag-up: what-what; what-where; where-where
These parse into what and where, from which actions must be selected. Typical
actions associated with click-and-drag
- multiple triggers with differentiated meanings: multiple buttons or
- chorded clicking: only order matters
- rhythmic clicking: real-time interface
- Operate, works fine as long as there is a single obvious operation that
you want to do
- Open a folder
- Edit a document
- Execute an application
- Why is double-clicking the interface? (one click to select, one for
the default command?)
- Delete can be considered a special case of operate. There is an
argument that it should be done slowly, except when it's part of a
- Create, applies to anything: need to know what to create, and where to
- what-where, metaphor is a well, draw items from an infinite supply,
and put them in the indicated positions
- where-what, metaphor is parachute, one position, a variety of
- where-where, metaphor is container that moves with the cursor
- what-what, see other operations, below.
- Other operations
- drag to an object that does the operation
- e.g., drag a file to the printer; drag a file to the compiler (Drag
two files to the compiler: a file of flags and one of code? Also
there will be rules for where the output goes, and what its name will
- e.g. data-flow style of systems, such as piping in the Unix
- drag from one icon to another
- operates by having special area on icon
- once you have one special area you can have many
- one for standard input, one for standard output, another for
- visual programming languages