CS251 - Computer Organization and Design - Spring 2008

Lecture 32 - Cache Misses

Practical Details

1. Assignment 7
2. Finished pipelined execution

Cache Concepts

How the big picture is implemented

Break the address into pieces

• High order bits determine which block
  • low order bits determine where in the block
• Where the break is done determines the number of blocks and the size of the blocks
• Here the block is the size of a cache line

Effect is that the low order bits apply to many locations in memory.

The cache normally contains an integral number of lines, usually a power of 2.

• Break the block number into two pieces
  1. low order bits adequate to address each line in the cache
  2. high order bits indicate which block in memory is in that line.
• Store the high order bits with the line

When a memory reference occurs

1. Break the address into three pieces
   1. address in the line
   2. address of the line
   3. high order bits of the block number
2. Using the line address retrieve the high order bits of the block number.
3. Compare to the address
4. If they match
   1. read from or write to the address
   2. on write hardware usually also writes the main memory asynchronously using a write buffer. This is called write-through.
   3. It is also possible to re-write memory only when the cache line is replaced. This is called write-back.

   otherwise

   1. Stall the processor
   2. Use the block number to get the relevant block from memory
   3. When it is installed, rerun the instruction

This way of doing things is called direct mapping.

Examples of Cache

1. One word per cache line, 16 Cache lines

Cache

Address

Circuit (p. 476)

1. Ignore access size bits
2. Use 5-2 to choose the line in the cache
3. Test 31-6 against the tag in the line
4. AND with valid to get HIT
5. Return data in line plus HIT

Comments

A very basic cache.

Easy to see how to extend to a bigger cache.

2. Longer cache lines: 16 word line, 256 lines

Cache

Address

Circuit (p. 486)

1. Ignore bits 0-1
2. Use bits 13-6 to choose the line in the cache
3. Compare bits 31-14 to the tag
4. AND the result with Valid to get HIT
5. Multiplexer selects word to return based on bits 5-2

Comments

Typical early cache technology

Cache Misses

On Read

Send memory line number to main memory (SDRAM).

SDRAM returns whole line in one transaction

• Address to DRAM, Data 0 to Cache, Data 1 to cache, etc.
• Wait until word 0 typically 40 nsec
• Then time per word much faster, 2-4 nsec.
• Reason is slow row addressing, fast column addressing

Load instruction, or instruction fetch is rerun.

• Pipeline is frozen while this takes place

On Write

Send memory line number to main memory (SDRAM).

SDRAM returns whole line in one transaction

• Address to DRAM, Data 0 to Cache, Data 1 to cache, etc.
• Wait until word 0 typically 40 nsec
• Then time per word much faster, 2-4 nsec.

Store instruction is rerun.

• Pipeline is frozen while this takes place.

Main memory is synchronized

• Write-through: two writes happen at once
  • but what if another write occurs immediately after?
  • possibly stall
  • assisted by a hardware write queue
• Write-back: don't synchronize immediately, but when the cache line leaves the cache
  • some misses are even more expensive
  • multi-processing is difficult

Possible Improvements

Obvious trade-off: the longer the lines

• the fewer misses, and

  the higher the cost of a miss

The best performance depends on locality in the code.

• the compiler can help.

Typically, cache misses slow the processor by a factor of 2.

Widen and speed up bus between main memory and cache.

Send the desired word first, then fill the rest of the line.

Return to:

• Bill Cowan's lecture notes for CS251 in W08
• Bill Cowan's Winter 2008 CS251 page
• Bill Cowan's CS251 page
• Bill Cowan's teaching page
• Bill Cowan's home page