CS457 - System Performance Evaluation - Winter 2010


Public Service Announcements

  1. Mid-term

Lecture 12 - Data Analysis II

Motivation of Analysis of Variance (pdf)

Example - Reponse Time

Preparing the data

Looking at the Data

Testing the Tentative Conclusions

For example

  1. Histogram of response times made the response time seem to be about 12 msec.
  2. Calculate the average
  3. Remember your statistics and calculate the sample variance and standard deviation.
  4. Look back at at your tentative conclusions. For example,

When your data has passed the tests

Measurement is finished. How do you use the results?

Example. You saw a difference of 20 msec between response time for reqtype1 (browsing) and reqtype2 (searching).

Example. You saw that each increment of 1 in the load makes an increment of 500 msec in the response time for browsing.

Analysis of Variance (ANOVA) aka Linear Models

Zero Factor Analysis of Variance

New Concepts

General idea

  1. Here is a list of ten genuinely random 2-digit numbers
  2. Here is a list of ten genuinely random 2-digit numbers with 15 subtracted from each
  3. In each case we ask whether the reduction in the variance (15.3 in the first case, 313.3 in the second) is statistically significant.
  4. To work it out from first principles
    1. Assume underlying distribution from which data is drawn has finite variance.
    2. N data points
    3. Mean is normally distributed
    4. Variance is distributed by a chi-square distribution with N degrees of freedom
    5. The treatment variance, the difference between two variances, is distributed by a chi-square distribution with N1=1 degrees of freedom.
    6. Ratio between the treatment variance and the remaining error variance is distributed by an F distribution with N-1 and N1 degrees of freedom
    7. Check the ratio (15.3 * (N-N1)/ 541.4 * N1 in the first case, 1381.3 * (N1-1) / 1068.0 * N1 in the second) against the percentage points of the F distribution
  5. Of course, for a test as simple as this there are other, and better, ways of doing the test.

One Factor Analysis of Variance

New Concepts

The Linear Model

The Calculation

Two Factor Analysis of Variance

New Concepts

The Linear Model

The Calculation

.


Return to: