CS457 - System Performance Evaluation - Winter 2008

Questions and Comments

1. Tutorial: MC4045, Monday March 10, 18.00 to 19.00

Lecture 25

Text: chapter 28 & 29

Continuous Random Variables

2. Exponential distribution

Use for infinite populations that produce finite event rates.

1. f(x) = \lambda exp(-\lambda*x)
2. F(x) = 1 - exp(-\lambda*x)
3. u = F(x)

   exp(-\lambda*x) = 1 - u

   x = -log(1-u) / \lambda = -log(u) / \lambda

   But, remember that there is a special case! (u = 0)

5. Normal distribution

Use when you want to add noise

1. f(x) = exp(-(x^2)/2*\sigma^2)
2. No easy form for F(x).
3. But, f(x,y) = exp(-(x^2 + y^2) / 2*\sigma) has a simple F(x,y), which I won't work out.
4. Thus, you choose two uniform variates, u1 & u2.
5. And you get back two independent random numbers
   • x1 = \sigma*cos(2\pi*u1)* sqrt(-2log(u2))
   • x2 = \sigma*sin(2\pi*u1)* sqrt(-2log(u2))

Random Number Generators - Pseudo-random Numbers

Text: Chapters 26 & 27

What we want

1. Computationally efficient
2. Gives good sequences of numbers
   • uniform
   • independent
3. Long periods
4. Reproduce previous sequences for debugging

Linear Congruential Generators

X(n) = ( aX(n-1) + b ) mod m

• a, b, m are constants
  • The quality of the sequence depends strongly on the values chosen for a, b & m.
  • Period is a divisor of m
• X(0) is the seed
• u(i) = X(i) / m has the correct range.

How to get a full period - all possible values appear before repeating.

• GCD( m, b ) = 1
• Every prime factor of m is a factor of a-1.
• If 4 divides m the 4 must divide a-1

Choose m = 2^k for efficiency.

• Is this really true in 2008?

Testing Sequences of Random Numbers

Serial test

1. Get adjacent pairs of numbers as points in a plane.
2. Put them into bins: kxk.
3. If sequence is uniform and random then P(point is in a given cell) = 1/k^2
4. Use Kolmogorov test (or chi-square test) to check the distribution.

Gap test

1. Choose an interval between zero and one, of length p
2. Then P( U in interval ) = p
3. Look at runs of points outside interval
   • P( run of 0 ) = p
   • P( run of 1 ) = p(1-p)
   • etc.

   This is a geometric distribution

Practical Issues

Two Phases of a Simulation

Transient

Steady State

Look for a "knee"

Testing Paired Samples

Text: Section 13.4.1

This is a technique for getting more significance from a given set of runs. It requires that you design the experiment correctly.

Comparing two alternative configurations

1. Create a complete design
   • the two configurations crossed with all other relevant variables

   Pair trials that are the same in all respects, except for configuration

   Compute the differences in performance metric between the pairs.

   Test it against zero using a T-test.

How Long Should a Simulation Run?

How many replications should there be?

1. Perform n replications
2. Discard data from transient phase
3. Calculate performance metric
4. Result is m values {y1, y2, ... , yn}, the sample
5. Calculate mean, m, and variance, s, of sample
6. x% confidence interval is m \pm T(n-1, x%) (s/n^(1/2))
7. Select n for the real runs so that you are sure the confidence interval will exclude zero.

Return to:

• Bill Cowan's Winter 2008 CS457 page
• Bill Cowan's CS457 page
• Bill Cowan's teaching page
• Bill Cowan's home page