CS349 - Implementing User Interfaces - Spring 2010

Public Service Annoucements

1. Assignment 3.

Lecture 27 - Geometry

The Future of Direct Manipulation

Lasso is distinctly different from anything we have seen so far.

• How?

Overloading

Gesture

Multitouch

Touch the screen in more than one place, like playing a chord on the piano.

• Some independent motion of the fingers is possible.
• Fingers are naturally big, compared to the screen.
  • good at relative motion
  • cannot do absolute motion without landmarks

Learning The Principles of Direct Manipulation

1. Geometry manipulation applications have the richest repertoire of direct manipulation.
2. Direct manipulation requires a more powerful model of geometry.

A Preliminary to Geometry - Points and vectors

A point is a location in space; what is the difference between two points?

• a vector

Both are designated by two real numbers (in the plane); what's the difference?

• consider translations in the plane
  • points change
  • vectors don't
• Computer example - addresses and offsets
  • offsets are the difference between addresses
  • create position independent (relocatable) code by using only offsets.

Thus, we have two kinds of entities, both represented by the same notation. To differentiate

• (x, y, 1) is a point.
• (x, y, 0) is a vector.

Does this make sense?

1. P - P -> V
2. P + V -> P
3. V + V -> V
4. P + P doesn't make sense: it gives (x + x', y + y', 2), which isn't defined
5. aV -> V
6. aP doesn't make sense: it gives (ax, ay, a), which is only defined if a=1..

Important finicky (and practical) point

• The interface we create is, in theory, position independent, but
  • to appear on a screen it must be attached to a particular position.
• Thus, when we write
  • DrawRectangle( window, x, y, width, height );

  window - indicates the origin to use

  x, y - is a point at which to start drawing

  width, height - define vectors that are combined with the starting point to make new points

Graphical User Interfaces

What does this tell us about graphical user interfaces (more precisely, interfaces that use direct manipulation)?

• doing graphics
  • separate DrawRectangle into two parts.
    1. a point part to place the first vertex
    2. a vector part to get from the first vertex to each subsequent vertex
  • separate `click and drag' into two parts
    1. a point part to locate the object to be dragged
    2. a vector part to update geometry while dragging occurs
  • make moving the object a vector operation, which depends on the position of a single reference point
    • usually the upper left corner of the enclosing rectangle
    • moving the object simply moves the reference point
    • implemented in all graphics hardware as bitblt
• analysing input
  • Where a gesture starts is a point.
    • When the tracker is sampled the n'th time, the vector from sample n-1 to sample n is added to the description of the object.
    • The we have its location, the first point, separated from the object, the vectors, which is translationally invariant.
    • Remember the special properties of invariances like
      • translation,
      • rotation
      • scaling
      for humans.
  • Getting gestures in a well-defined geometry means that we can apply them directly to stuff on the display.
    • The next few sections show how to do it.

We (for some value of we) know a powerful formalism for manipulating the vector part

• linear algebra
  1. rotation
  2. scaling
  3. reflection

  But, remember that these transformations all occur with respect to the origin, which you can consider to be the tail of the vector

• represented as 2x2 matrices, which operate on two-dimensional objects. But we are writing a vector not as (x, y), but as (x, y, 0), so we need to operate on three dimension objects.
  • We must extend the matrices to 3x3 in order to handle the third coordinate correctly.
  • Consider the structure of matrix multiplication. The upper left 2x2 part is just a two-dimensional matrix operating on a two-dimensional vector (x, y).
  • The first two elements of the third row just multiply the third component, which is zero, so they can be anything.
  • All vectors (x, y, 0) must have a scalar product of zero with the third column, which must be (0, 0, ?).

  So, getting the right result when we transform a vector determines the first two rows of the transformation matrix. The remainder of the matrix is determined by getting points right.

How do we extend these to points?

1. Rotate about the origin.
2. Reflect about the y axis.
3. Scale with respect to the origin.

To get these correct in 3x3 matrices

• bottom row must be (0, 0, 1)
• For example, the effect of rotating the point (x, y, 1) about the origin by the angle t is

          (  cos(t)  sin(t)  0 )
(x, y, 1) ( -sin(t)  cos(t)  0 ) = (x cos(t) - y sin(t), x sin(t) + y cos(t), 1) 
          (    0       0     1 )

Now we just need to add in translation.

1. Leave vectors unchanged (x, y, 0) -> (x, y, 0)
   • 2x2 identity matrix
   • rightmost column must be (0, 0, anything)
2. Translate points (x, y, 1) -> (x+dx, y+dy, 1)
   • bottommost row must be (dx, dy, 1)

Now we can get any geometric figure specified as sets of points and vectors anywhere and any shape, just by multiplying matrices, which computers are very good at.

Matrix multiplication is just a technology for implementing an interface, and not an interface. We also need a framework for putting a human-understandable interface onto matrix multiplication. Here's how it's done.

1. Implement an interface that returns a geometric operation that you want to perform.
2. Interpret the geometric operation in terms of a sequence of basic transformations
   • rotating around the origin
   • scaling with respect to the origin, along the x-axis
   • reflecting in the y axis
3. Write each basic transformation as a matrix.
4. Multiply the matrices together, and apply the result to the thing you want to transform

Redundancy check

There are two ways to rotate a rectangle which is away from the origin.

1. Consider it as a collection of points
   1. Translate to the origin
   2. Rotate at the origin
   3. Translate back to where it started
2. Consider it as one point plus a collection of vectors
   1. Rotate the vectors

They `obviously' give the same answer. (Exercise left to the reader.)

You will find both representations in PostScript because one is sometimes more convenient than the other.

But for making a graphical interface it's usually better to stick to one or the other.

Graphical User Interfaces

Last lecture we talked about points and vectors, how they transform.

How does this help us make graphical user interfaces (more precisely, interfaces that use direct manipulation)?

Let's think for a minute about assignment 5. What needs to be done?

1. drawing
   • using painter's algorithm and Xlib - need to know parameters like x, y, width, height
2. selecting
   • depends on an inside test
3. transforming
   • translate
   • rotate
   • rescale
   • reflect

Here's how we do it.

1. Base all things you draw on a small number of basic figures, such as
   • a standard line running from (0, 0) to (1, 0)
   • a standard rectangle with corners at (0, 0), (0, 1), (1, 1), (1, 0)
   • a standard ellipse with zero eccentricity (a circle), centred at (0, 0), of radius 1.
   • and so on.
2. Use transformations to turn this into any quadrilateral shape
   • represent transformations by matrices 3x3
     1. translation by dx, dy
        • point: (x, y, 1) -> (x + dx, y + dy, 1)
        • vector: (x, y, 0) -> (x, y, 0)
        • Matrix that does this must do its thing in the bottom row
     2. rotation by t
        • point: (x, y, 1) -> (x cos(t) - y sin(t), x sin(t) + y cos(t), 1)
        • vector: (x, y, 0) -> (x cos(t) - y sin(t), x sin(t) + y cos(t), 0)
        • Matrix that does this must do its thing in the 2x2 part.
     3. reflect about y-axis
        • point: (x, y, 1) -> (-x, y, 1)
        • vector: (x, y, 0) -> (-x, y, 0)
        • Matrix that does this must do its thing on the diagonal of the 2x2 part
     4. rescale by a along the x-axis
        • point: (x, y, 1) -> (ax, y, 1)
        • vector: (x, y, 0) -> (a, y, 0)

        Matrix that does this must do its thing on the diagonal of the 2x2 part

Matrices

The matrices you need (transposed if you like column vectors rather than row vectors)

1. Translation (points only)

   ( 1.0    0.0   0.0 )
   ( 0.0    1.0   0.0 )
   ( dx     dy    1.0 )

2. Rotation about the origin

   (  cos(t)   sin(t)   0.0 )
   ( -sin(t)   cos(t)   0.0 )
   (    0.0      0.0    1.0 )

3. Scale along the x-axis

   (   a    0.0    0.0 )
   ( 0.0    1.0    0.0 )
   ( 0.0    0.0    1.0 )

4. Reflect about the y-axis

   ( -1.0    0.0    0.0 )
   (  0.0    1.0    0.0 )
   (  0.0    0.0    1.0 )
   
       

These are standard transformations. How do we do more interesting and useful work?

• Make the transformations we want as compositions of the basic transformations
• Examples.
  1. Rotation about an arbitrary point.
  2. Scaling along an arbitrary direction.
  3. Scaling about an arbitrary point and along an arbitrary direction.
  4. Reflection in an arbitrary line.

These matrices transform points and vectors. Let's see how we do drawing.

1. Store the standard shape in terms of points and vectors.
2. Know how to draw the standard shape in terms of its points and vectors.
3. Store with it a transformation matrix that turns it into the actual shape.
4. Draw the actual shape using the transformed points and vectors.

And let's see how we do the inside test. (We could just use the transformed points, but there's an easier way to do it.)

1. Fact. The transformations above - when applied to a point and a shape - do not affect whether or not the point is inside the shape.
2. This gives us two options for doing the inside test
   1. Transform the standard shape into the point's world, using the transformation matrix stored with the point.
   2. Transform the point into the standard shape's world, using the inverse of the transformation matrix.

   We prefer the second, because we can choose the standard object so that the inside test is easy.

Therefore store the inverse matrix with the shape, along with the transformation matrix.

Exercises.

1. What is the inverse matrix of each of the basic matrices?
2. How do we multiply inverse matrices?

Return to:

• CS349 - Spring 2010 - Lecture schedule
• CS349 - Spring 2010
• CS349 page
• Bill Cowan's home page.