CS457 - System Performance Evaluation - Winter 2008

1. Assignment 1

2. E-commerce example

Lecture 5 - Performance Metrics

Creating an Abstraction

Section 2.2: 1-5 & 7

Two example systems

  1. RPC versus Remote pipes
  2. E-commerce server
Stage RPC E-commerce
1. Define Goals, System
  • goals first
  • minimize system size
 Write a paper comparing RPC to RPs
  • System should include overhead.
  • System should not include computation
 Ensure that a system architecture is scalable.

Find the right size for an expected workload
2. List services (from 1)
  • possible outcomes for each service
  • including failures
 Data transfer (why?)
  • large transfers
  • small transfers
  • failures omitted (why?)
 Catalogue updates

Client requests

  • too many to list
  • make generic categories
3. List metrics (from 2)
  • speed
  • accuracy
  • utilization & availability
 Speed is everything. (The measurements are performed on an unloaded system with errors discarded.)
  • throughput
  • average response time
  • maximum request rate
 speed
  • average response time
  • maximum response time

accuracy

  • incorrect transactions
  • inconsistencies
4. List parameters (from 2, 3)
  • that affect performance as defined by metrics
  • system parameters
    • h/w & s/w resources
    • resource characteristics
    • system configuration
  • workload parameters
    • determined by client objectives
 System
  • client speed
  • network speed
  • server speed

Workload

  • request rate
  • request size
 System
  • all the ways the system can vary
  • especially internal failures

Workload

  • rate of requests
  • number of simultaneous clients
5. Select factors to study (from 1,4)
  • determined by study goals
  • What about the other parameters?
 System
  • network delay

Workload

  • request size
6(7). Select workload (from 2)
  • measurement: scripted, usually with randomness, could be a trace or log
  • simulation: scripted or trace
  • model: probability distribution

Metrics

Sequence of events for a single request

  1. System receives request, ai
  2. System starts executing the request - reaction time
  3. System starts responding to request - response time 1
  4. System completes response to request, di - response time 2, ri = di - ai

Types of metrics

  1. Speed
  2. Accuracy
  3. Availability

Precise definitions of some common metrics.

System description to be read before Friday.

Return to: