CS789, Spring 2007

Lecture notes, Week 10

UI Metaphors

Why do they exist?

How do they function?

• learning transfer by way of
  • model-building
  • story telling

Fundamental to user experience

• Newton's law of inertia, with friction
  • things stay where you leave them
  • things don't change without a cause
• Conclusion, changes of position are significant.

Icons and Direct Manipulation

Separate dialogue styles exist along a continuum: linguistic-->graphical, menus and forms in the middle. Linguistic is based on words, talking:

• auditory, serial, temporal
• "You can't talk and listen at the same time."
• alternates listening and talking in significant chunks
• compare feedback in OS interaction to conversational feedback

• visual, parallel, spatial
• "You can see (active), monitor (passive), and act at the same time." (There are even background actions.)
• continuous monitoring of the activity
• metaphor includes something that moves continuously toward the goal

• In each case an action triggers the disposal of the file: carriage return at the end of a command line, releasing the mouse button with item over the high-lighted trash, releasing the mouse button with "delete" highlighted.
• In each case there is real progress toward a goal,
• but the feedback is very different: lengthening string of characters, increasing nearness of icon to trash, progress in selecting icon & menu command activation.

1. commands with names as objects
2. commands with icons as objects
3. icons as commands and as objects.

General organization of graphical dialogue styles:

• Graphical symbols (icons) designate objects. Symbols have a spatial location, on the display surface.
• Important distinction is discrete versus continuous (nominal/ordinal versus difference/ratio):
  • icons: static graphical representation, discrete set of underlying objects
  • position: continuous
  • graphical objects: variable graphical representation, continuous set of underlying objects
• Let us distinguish direct manipulation from graphical interfaces.
  1. direct manipulation
     • icons, each of which has a position, and a reference
     • icons are unchanging; icon characteristics are only a convenience for the user, helping the user make the correct reference. For example
       • On a bank machine, instead of "WITHDRAW CASH",
       • we might have a pile of cash inside an account icon, and
       • we drag the cash to wallet icon.
       • Colour of the wallet probably isn't significant.
       • What about the colour of the bills?
     • locations of icons change; relative location indicates operations to be performed
  2. graphical interfaces
     • graphical objects, each of which has a position, and a lot of other properties
     • the other properties of graphical objects can be manipulated, most often as they are because the screen object is actually a picture of the real object
     • locations of objects also matters, and makes possible a complex object/subobject structure.
• We need to define and exploit possible interface operations. For example,
  1. point-and-click
     • empty space: create
     • on an icon: select
     • on part of a graphical object: select
  2. point-and-double-click
     • empty space: ??
     • on an icon: open/execute
     • on part of a graphical object: expand (show sub-parts)
  3. point-click-and-drag
     • starting in empty space: area for multiple selection
     • starting on an icon: move the icon, end-point dependent operation
     • starting on part of a graphical object: change parameter(s) of the object
  The problem here is that there are insufficient operations to exploit: the mouse buttons have insufficient bandwidth to occupy our full attention. We want the amount of activity generated by our motor systems to match the bandwidth of our perceptual systems to monitor it, and the bandwidth of our cognitive systems to keep track of it.
• Here are three ideas for increasing the number of things we can do directly
  1. shift-point-and-click, control-point-and-click, etc.
  2. point-and-click-B1-then-B2, point-and-click-B3, etc.
  3. palette with different cursors that change the meaning of point-and-click.
  Have you seen any of these interaction techniques? Why do you think they are used so little?

In the end these interaction techniques show

• current limited use of direct manipulation techniques
• substantial promise of direct manipulation techniques
• chicken and egg problem analogous to how you get from bacterial purple to the eye.

Icons

What is an icon?

1. graphical name
2. names an object known to the interface
3. its internal structure is not directly accessible, but is often indirectly accessible through a form
4. identity, location
5. subject to a limited set of operations

Why use icons instead of words?

1. help the user to remember: icons can even be animated. This points out the interesting double existence of icons.
2. visual distinctiveness: a richer set of perceptions related to finding, arranging, grouping.

What factors make icons visually distinctive? Distinctiveness is a property of a set of icons!

1. colour
2. some aspects of shape: angles, lines, curves
3. usually not conjunction or disjunction
   • Select one of several distinctive dimensions.
   • Processing more than one dimension is hard.
4. your eye will make a good decision
5. you can always do simple tests
   • Can you see the pattern?
   • Is search time independent of distractors?

   but it's likely that you care about peripheral distinctions, not foveal ones.

Select-Position

How are selection actions carried out by the user?

1. single
   • click in the space occupied by the icon: "Click on the icon."
   • multiple select one
2. multiple: indicate space containing several icons
   • Draw out the space
   • Text is a linear space: indicate beginning and end.
   • Take advantage of granularity: spreadsheet, text
   • Are there useful granularities for other objects in the interface? rows, columns, clusters, everything?
3. expand or contract
   • toggle, as in shift-click
   • new end-point, for linear structure
   It's interesting to note that discontinuous selection, which seems to be a hard interface to produce for linear structures, is a fact of life for unstructured collections. Why?
4. differentiated selection
   • multiple, differentiated arguments
   • feedback is what's hard, showing the user the current state of selection

Why do users change the placement of icons?

• for multiple selection
• for visual clarity

• two arguments: what to place, where to place it
• what by selection
• where by selection
• how prominent should the path be?

• containment, files directories - relies on memory
• icons with common properties - relies on perception

Commands

How do we indicate commands to be carried out?

• Select object of command; give command
• selection indicates the arguments
• How are commands given?
  • menu
  • mouse action, such as double-click
  • hot key on keyboard
  • special destination of a drag (move command)

• differentiated multiple selection for multiple arguments: exchange selection 1 and selection 2
• icon identity: delete all objects with icons like the selected one
• drag path: print then delete

Direct Manipulation

Differentiate interface operations (with pointer) from the actions they entail (create). Typical operations

1. position: where or what
2. click: trigger
3. down-drag-up: what-what; what-where; where-where

1. multiple triggers with differentiated meanings: multiple buttons or keyboard modifiers
2. chorded clicking: only order matters
3. rhythmic clicking: real-time interface

1. Operate, works fine as long as there is a single obvious operation that you want to do
   • Open a folder
   • Edit a document
   • Execute an application
   • Why is double-clicking the interface? (one click to select, one for the default command?)
   • Delete can be considered a special case of operate. There is an argument that it should be done slowly, except when it's part of a game interface!
2. Create, applies to anything: need to know what to create, and where to create it
   • what-where, metaphor is a well, draw items from an infinite supply, and put them in the indicated positions
   • where-what, metaphor is parachute, one position, a variety of items
   • where-where, metaphor is container that moves with the cursor
   • what-what, see other operations, below.
3. Other operations
   • drag to an object that does the operation
   • e.g., drag a file to the printer; drag a file to the compiler (Drag two files to the compiler: a file of flags and one of code? Also there will be rules for where the output goes, and what its name will be.)
4. Joining
   • e.g. data-flow style of systems, such as piping in the Unix shell
   • drag from one icon to another
   • operates by having special area on icon
   • once you have one special area you can have many
     • one for standard input, one for standard output, another for standard error
   • visual programming languages

Graphical Interaction

Return to:

• Bill Cowan's home page.
• Bill Cowan's teaching page.
• Bill Cowan's CS789 page.
• Bill Cowan's CS789 page for Spring 2007.