CS457 - System Performance Evaluation - Winter 2008


Questions and Comments

  1. Tutorial: MC4045, Monday March 10, 18.00 to 19.00

Lecture 25

Text: chapter 28 & 29

Continuous Random Variables

2. Exponential distribution

Use for infinite populations that produce finite event rates.

  1. f(x) = \lambda exp(-\lambda*x)
  2. F(x) = 1 - exp(-\lambda*x)
  3. u = F(x)

    exp(-\lambda*x) = 1 - u

    x = -log(1-u) / \lambda = -log(u) / \lambda

    But, remember that there is a special case! (u = 0)

5. Normal distribution

Use when you want to add noise

  1. f(x) = exp(-(x^2)/2*\sigma^2)
  2. No easy form for F(x).
  3. But, f(x,y) = exp(-(x^2 + y^2) / 2*\sigma) has a simple F(x,y), which I won't work out.
  4. Thus, you choose two uniform variates, u1 & u2.
  5. And you get back two independent random numbers

Random Number Generators - Pseudo-random Numbers

Text: Chapters 26 & 27

What we want

  1. Computationally efficient
  2. Gives good sequences of numbers
  3. Long periods
  4. Reproduce previous sequences for debugging

Linear Congruential Generators

X(n) = ( aX(n-1) + b ) mod m

How to get a full period - all possible values appear before repeating.

Choose m = 2^k for efficiency.

Testing Sequences of Random Numbers

Serial test

  1. Get adjacent pairs of numbers as points in a plane.
  2. Put them into bins: kxk.
  3. If sequence is uniform and random then P(point is in a given cell) = 1/k^2
  4. Use Kolmogorov test (or chi-square test) to check the distribution.

Gap test

  1. Choose an interval between zero and one, of length p
  2. Then P( U in interval ) = p
  3. Look at runs of points outside interval

    This is a geometric distribution


Practical Issues

Two Phases of a Simulation

Transient

Steady State

Look for a "knee"

Testing Paired Samples

Text: Section 13.4.1

This is a technique for getting more significance from a given set of runs. It requires that you design the experiment correctly.

Comparing two alternative configurations

  1. Create a complete design

    Pair trials that are the same in all respects, except for configuration

    Compute the differences in performance metric between the pairs.

    Test it against zero using a T-test.

How Long Should a Simulation Run?

How many replications should there be?

  1. Perform n replications
  2. Discard data from transient phase
  3. Calculate performance metric
  4. Result is m values {y1, y2, ... , yn}, the sample
  5. Calculate mean, m, and variance, s, of sample
  6. x% confidence interval is m \pm T(n-1, x%) (s/n^(1/2))
  7. Select n for the real runs so that you are sure the confidence interval will exclude zero.


Return to: