CS457 - System Performance Evaluation - Winter 2008

1. Textbook

Lecture 3 - Performance Metrics

Topics in Performance Evaluation

1. Abstract: based on objectives
   • Abstractions are good or bad
   • But the good ness and badness is not measurable, at least not ex ante
2. Design: what to measure
   • More measurements are only sometimes better
   • recursive performance analysis

   Design also applies to simulation and to modelling.

3. Measure: good measurements produce robust data
   • `robust' means `If you did it again holding only the abstraction constant you would get the same results.'
4. Analyse: data requires objective analysis to produce results. Because
   • You are human, and
   • Humans are half-blind. (All too often we see what we want to see.)
5. Simulate: results require surface explanations
   • Surface explanations solve immediate problems
6. Model: surface explanations sometimes demand deep explanations
   • Deep explanations actually make you, and those you work with, smarter.
   • Remember that models must be tested.
   • There is no `waterfall model' in performance evaluation.

Creating an Abstraction

Section 2.2: 1-5 & 7

Two example systems

1. RPC versus Remote pipes
   • Simple, but easy to understand
2. E-commerce server
   • Realistic, but easy to get swamped by the details

Metrics

Sequence of events for a single request

1. System receives request, ai
2. System starts executing the request - reaction time
3. System starts responding to request - response time 1
4. System completes response to request, di - response time 2, ri = di - ai

Types of metrics

1. Speed
   • response time
   • throughput
     • mean response time = L/n = 1/throughput
2. Accuracy
   • error probability
3. Availability
   • uptime/downtime
   • utilization

Return to:

• Bill Cowan's lecture notes for CS457 in W08
• Bill Cowan's Winter 2008 CS457 page
• Bill Cowan's CS457 page
• Bill Cowan's teaching page
• Bill Cowan's home page