how do you find the miss penalty of a single level cache?

I am attempting to find the average memory access time (AMAT) of a single level cache. In order to do so, miss penalty must be calculated since the AMAT formula requires it. Doing this for a multilevel cache requires using the next level cache penalty. But for a single level, there is obviously no other cache level. So how is this calculated? formula: AMAT = HIT-TIME + MISS-RATE * MISS-PENALTY

1 Answer 1

You have the correct formula to calculate the AMAT, however you may be misinterpreting the components of the formula. Let’s take a look at how to use this equation, first with a single-level cache and next with a multi-level cache.

Suppose you have a single-level cache. Hit time represents the amount of time required to search and retrieve data from the cache. Miss rate denotes the percentage of the data requested that does not reside in the cache i.e. the percentage of data you have to go to main memory to retrieve. Miss penalty is the amount of time required to retrieve the data once you miss in the cache. Because we are dealing with a single-level cache, the only other level in the memory hierarchy to consider is main memory for the miss penalty.

Here’s a good example for single-level cache: L1 cache has an access time of 5ns and a miss rate of 50% Main memory has an access time of 500ns AMAT = 5ns + 0.5 * 500ns = 255ns 

You always check the cache first so you always incur a 5 ns hit time overhead. Because our miss rate is 0.5, we find what we are looking for in the L1 cache half the time and must go to main memory the remaining half time. You can calculate the miss penalty in the following way using a weighted average: (0.5 * 0ns) + (0.5 * 500ns) = (0.5 * 500ns) = 250ns .

Now, suppose you have a multi-level cache i.e. L1 and L2 cache. Hit time now represents the amount of time to retrieve data in the L1 cache. Miss rate is an indication of how often we miss in the L1 cache. Calculating the miss penalty in a multi-level cache is not as straightforward as before because we need to consider the time required to read data from the L2 cache as well as how often we miss in the L2 cache.

Here’s a good example: L1 cache has an access time of 5 ns and miss rate of 50% L2 cache has an access time of 50 ns and miss rate of 20% Main memory has an access time of 500 ns AMAT = 5ns + 0.5 * (50ns + 0.2 * 500ns) = 80 ns 

Again, you always check the L1 cache first so you always incur a 5 ns hit time overhead. Because our miss rate is 0.5, we find what we are looking for in the L1 cache half the time and must down the memory hierarchy (L2 cache, main memory) the remaining half time. If we do not find the data in the L1 cache, we always look in the L2 cache next. We thus incur a 50 ns hit time overhead every time we miss in the L1 cache. In the case that the data is not in the L2 cache also (which is 20% of the time), we must go to main memory which has a memory access time of 500 ns.