# CS457 - System Performance Evaluation - Winter 2010

## Public Service Announcement

1. pdf available
2. Markovian service times

# Birth & Death Processes

#### Derivation

• See previous lecture.
• See pdf

#### Basic Result

dp_j(t) / dt = \lambda_j-1 p_j-1(t) + \mu_j+1 p_j+1(t) - (\lambda_j + \mu_j) p_j(t)

1. pn = p0 (l_(n-1)*...*l1*l0 / m_n *...*m1)
2. Set p0 by the condition sum_n p_n = 1.

## Example 3: Two Servers versus One Server Twice as Powerful

### One Server Twice as Powerful

#### Service rate and time:

• service rate: \nu = 2 \mu
• service time 1 / \nu = 1 / (2\nu)
• s = r / 2

#### Number of requests

• E[N] = s / (1-s) = r / (2-r)
• This is not quite as good as half as many. Why?

#### Response Time

• R[N] = 1 / (\nu - \lambda) = 1 / {\nu (1 - s)} = 1 / {2 \mu (1-r/2)} = 1 / {\mu (2-r)}
• This is not quite as good as twice as fast.

#### Utilization

• U = s = r/2
• Utilization is exactly one half, as we expect.

### Two Servers with One Queue

 Number of requests in the system, j 0 1 2 3 ... Probability of a birth, \lambda_j \lambda \lambda \lambda \lambda ... Probability of a death, \mu_j 0 \mu 2 \mu 2 \mu ...

#### Incremental Solution

• j = 0: \mu p_1 = \lambda p_0 ==> p_1 = p_0 * r, where r = \lambda / (\mu)
• j = 1: 2 \mu p_2 = ((\lambda + \mu) p_1 - \lambda p_0 ) / (2 \mu) ==> p_2 = p_0 * (1/2 r^2
• j = 2: 2 \mu p_3 = ((\lambda + \mu) p_2 - \lambda p_1 ) / (2 \mu) ==> p_3 = p_0 * (1/4) r^3
• ...
• j = n-1: 2 \mu p_n = ((\lambda + \mu) p_n-1 - \lambda p_n-2 ) / (2 \mu) ==> p_n = p_0 * (1/2)^(n-1) r^n

Now normalize,

• 1 = \sum_n p_n = p_0( 1 + 2 \sum_(n=1) (r/2)^n ) = p_0 ( 2 \sum_(n=0) (r/2)^n - 1 ) = p_0 ( 2 / (1-(r/2)) - 1 ) = p_0 ( 2 + r ) / ( 2 - r )
• p_0 = (2-r) / (2+r) = (2 \mu + \lambda) / (2 \mu - \lambda)

Thus, p_n = (2-r) r^n / (2+r)

#### Results

The mean number of jobs in the system is

• E[N] = sum_n n * (2 - r ) * r^n / (2+r) = {(2-r) / (2+r) } \sum_n n*(r/2)^n = 2*r / {(2-r)(2+r)} = r / {(2-r)(1+(r/2))}
• Compare this to E[N] = r / (2-r), the result for the twice as powerful server.
• We see the queue shrink by a factor of 1/(1 + (r/2))

Little's law

• Mean response time: E(R) = (1/\lambda) * E(N) = 1/ { \mu*(2-r)*(1+r/2) }
• Compare this to E[R] = 1 / {\mu (2-r) }
• We see the same response speed up by the same factor: 1/(1 + (r/2))

Utilization

• U = 1 - p_0 - 1/2 p_1= 1 - (2-r) / 2+r - (r/2) (2-r) / (2+r) = r / 2.
• Utilization does not change, as we expect.

## Example 5: Deterministic Service Times

Assume that the system is stable.

### State by State Analysis

• j = 0: Response time = m
• j = 1: Average response time = m + m/2
• j = 2: Average response time = 2m + m/2
• j = n: Average response time = nm + m/2

#### Average response time

E[R] = (m/2) * P(busy) + m * P(idle ) + \sum_j=1 p_j (jm )

P(busy) = \lambda * m

E[R] = (m/2) * (\lambda * m) + m * ( 1 - \lambda * m ) + m * E[N]

E[R] = (m/2) * (\lambda * m) + m * ( 1 - \lambda * m ) + m * \lambda * E[R]

E[R] = { (m/2) * (\lambda * m) + m * ( 1 - \lambda * m ) } / { 1 - m * \lambda }

E[R] = m { 2 - \sigma } / {2 * (1 - \sigma)}

E[N] = { (\sigma/2) * \sigma + \sigma * ( 1 - \sigma ) } / { 1 - \sigma }

• where \sigma = \lambda * m

E[N] = { \sigma * (2 -\sigma) } / {2 * (1 - \sigma)}

U = P(busy) = \sigma

#### Results

• \sigma -> 0: E[N] -> \sigma; E[R] -> m; U -> \sigma
• \sigma ~ 1/2: E[N] = 3/4; d(E[N])/d\sigma = 5/2; E[R] = 3m/2; U = 1/2
• \sigma -> 1: E[N] ~ 1/(1-\sigma); E[R] ~ 1/(1-\sigma); U -> 1

# General Markov Processes

### Solution method

1. Draw state transition diagram
2. Obtain balance condition equations: rate in equals rate out
3. Solve balance equations with normalizing equation
• \sum pn = 1
4. Determine performance metrics from pn

Simple in principle. May be complex in practice.

# Analytic Queueing Theory - Queueing Networks

Collection of interacting service providers.

One new concept:

• p_ij = P( job just finished at i will get immediate further processing at j ) 1 <= i,j <= M

## Operational Analysis

Very similar to what we did near the beginning of the course

• e.g., Little's law

### Open Networks in Steady State

Jobs enter the network and depart from it

1. Number of jobs is a random variable
2. External arrivals can appear anywhere
3. Extra destination for departures: p_i(M+1) = P( job departs from network immediately after service at i )

Definitions at server i

1. external arrivals \gamma_i
2. internal arrivals
3. total arrivals \lambda_i = \gamma_i + internal arrivals
4. throughput X_i = \lambda_i
5. total arrivals \lambda_i = \gamma_i + \sum_j X_j p_ji
• Remember: job can come back to where it just finished.

#### Solution procedure

1. Write balance equations
2. Solve balance equations
3. Derive performance metrics

Example: pdf

#### Utilization

U_i = \lambda_i s_i

• s_i, the mean service time, is a parameter assumed to be known

#### Throughput

X - rate of jobs departing from the network

1. X = \sum \gamma_i because the network is stable
2. X = \sum \lambda_i p_i(M+1)
3. Not too hard to prove that these are the same

### Closed Networks in Steady State

1. No external arrivals.
2. No departures
3. Number of jobs in the system, N, is constant

#### A. Single server per service centre

1. Total arrival rate

• \lambda_i = sum_j \lambda_j p_ji
• \sum_i p_ji = 1
• Balance equations give M equations, which are not linearly independent
• Can only solve for ratios of \lambda_i

Example: pdf

2. Utilization

• Can only solve for rations of U_i
• U_i / U_j = (\lanbda_i / \lambda_j) * s_i/s_j: the factor in paraenthesis is known

3. Throughput

Arbitrary decision

• Cut the system somewhere, e.g. at the arrival of server i
• Then X is the rate of jobs passing across the cut

#### B. Interactive System in Steady State

1. Call the set of terminals server 0

2. Throughput

Cut at entrance to terminals: X = \lambda_0

Define visit ratio: number of visits to service i per job

• V_i = \lambda_i / X = \lambda_i / \lambda_0

3. Utilization

U_i = \lambda_i s_i = X V_i s_i = X D_i

• D_i = X s_i is the demand per job at server i

U_i / U_j = D_i / D_j

4. Upper bound on throughput

D = \sum D_i

X(n): throughput with N users

• X(1) = 1 / (D + Z)
• X(N) <= N/(Z + D) : equality when there is no contention

5. Bottleneck analysis

1. b - server with highest demand
2. D_b = max{D_i} = D_max
3. U_b = X D_b <= 1
4. X <= 1/D_max

Therefore, X(N) <= min( 1/D_max, N / (D+Z) )

6. Lower bound on mean response time

1. Treat entire system as one server. Then R(N) = N/X - Z
2. R(N) >= D = sum_i D_i because X(N) <= N / (D+Z)
3. X(N) <= 1 / D_max, so that R(N) <= ND_max - Z

.