CS457 - System Performance Evaluation - Winter 2008

1. Metrics PDF

Lecture 8 - Baby Modelling

In Lecture 5, I asked you to read this system description. Today we will complete analysing it as an example of how to make a system abstraction.

We do this as methodically and mechanically as possible.

1. Study goal and System description

1a. Simplifications

2. Services Provided

3. Metrics

Load Balancing

1. Reponse to client requests
2. Delay in redirection
3. Utilization

Consistency

1. Time data is stale
   • catalogue item
   • shopping cart

Failure

1. System availability for login
2. Time to complete redirections

4. List Parameters

System Parameters

These describe system propertis that might be expected to affect the performance of the system in dimensions described by the goals of the study

1. network delay
2. frequency of transmission of shopping cart content
3. frequency of server failure
4. downtime on server failure
5. load balancing algorithm
6. service capacity: how many services a server can provide per second

Workload Parameters

1. arrival rate of requests of each type

Hybrid Parameters

1. service (response) time, which depends on service capacity and arrival rate of requests

5. Choose Factors

Falls under the heading of experiment design which, for now, remains outside the scope of system abstraction.

Choosing a factor to examine requires you to choose levels at which you want to know how the system performs.

Modelling

An extreme example of abstraction.

Two methods for modelling a system

1. Stochastic
2. Operational

1a. Study Goals

Get analytic results under as general a set of assumptions as possible. Results will be in the following areas:

1. Response Time
2. Throughput
3. Utilization

1b. System Description

1. A server that processes requests
2. A queue that holds requests currently being processed
3. A client that makes requests.

Note. `Request', these notes, and `job', textbook, are exactly the same thing for the purposes of this course. `Request' better characterizes what the client does and is both a noun and a verb, which is handy when writing and speaking; `job' better describes the actual thing that is requested and has a narrower range of meanings in English, which can reduce misunderstanding.

1c. Simplifications

YES!

2. Services Provided

Handling requests.

• A request is completely abstract: we are uninterested in what is requested.
• The only specified property of a request is the consumption of a set amount of resources during processing

3. Metrics

1. Statistics of response times, such as
   • average response time - R
   • median
   • minimum
   • maximum
   • percentiles
   • etc.
2. Statistics of throughput
   • average throughput - X
3. Statistics of utilization

4. Parameters

System Parameters

1. Service statistics

Workload Parameters

1. Arrival statistics
   • arrival rate - \lambda
2. Think time, which is really part of arrival statistics
   • average think time - Z
3. Number of users, which is really part of arrival statistics.

5. Factors

Same as parameters

Little's Law (or Formula)

1. Picture of arrival and departure generating system load diagram
2. Area under system load is total response time: nR
3. Average load, Q, times time interval, L is total response time: QL = nR
4. n/L = X = throughput = \lambda = interarrival time.

Stability

Little's formula holds only if the system is stable, which is defined by U < 0

6. Workload Models

1. Infinite population of clients
   • Is this actually possible? No, only in the limit.
   • Only one possible distribution for interarrival time.

     For a proof see this pdf.

   • Demonstration: binomial -> Poisson -> exponential
2. Finite population of clients - think-time model
   • client works (called thinking), then server works, then client works (called thinking), etc.

Applying Little's Formula in the Think-time Model

1a. Goal

1b. System Description

1c. Simplifications

2. Services Provided

3. Metrics

1. X - Throughput
2. R - average response time

4/5. Parameters/Factors

Return to:

• Bill Cowan's lecture notes for CS457 in W08
• Bill Cowan's Winter 2008 CS457 page
• Bill Cowan's CS457 page
• Bill Cowan's teaching page
• Bill Cowan's home page