Two-Level On-Chip Caches:

Norm Jouppi (Compaq), Steve Wilton

In recent years, increases in memory subsystem speed have not kept pace with the increase in processor speed, causing processor execution rates to become increasingly limited by the latency of accessing instructions and data. On-chip caches are a popular technique to combat this speed mismatch. As integrated circuits become denser, designers have more chip area that can be devoted to on-chip caches. Straight-forward scaling of cache sizes as the available area increases may not be the best solution, however, since the larger a cache, the longer its access time. Using cache hierarchies (two or more levels) is a potential solution. In this projects, we explored the tradeoffs in the design of on-chip microprocessor caches for a range of available on-chip cache areas.

Potential Advantages of two-level on-chip caching:

• The primary (L1) cache usually is split into separate instruction and data caches to support the instruction and data fetch bandwidths in modern processors. Many programs would benefit from a data cache that is larger than the instruction cache, while others would prefer the opposite. By having a two-level hierarchy on-chip, a majority of the cache lines are dynamically allocated to contain instructions or data.
• The cache access time is likely to be lower. As existing processors are shrunk to smaller lithographic feature sizes, the die area typically needs to be held constant in order to keep the same number of bonding pads. When processors are initially designed, their on-chip cache access times are usually well-matched to their cycle times. If the additional area available due to a process shrink is used to simply extend the first-level cache sizes, the caches will get slower relative to the processor datapath. Instead, if the extra area is used to hold a second-level cache, the primary caches can scale in access time along with the datapath, while additional cache capacity is still added on-chip.
• The second-level cache can be made set-associative while keeping the primary caches direct-mapped.
• Primary caches can be made less than or equal to the processor's virtual page size, allowing address translation to occur in parallel with the cache access.
• A chip with two-level cache structure will likely use less power, since most accesses would not involve the second-level cache.

Potential Disadvantages of two-level on-chip caching:

• If the total capacity of the first-level cache is not much smaller than that of the second-level cache, the second-level cache will consist of instructions and data which are already in the first-level cache. Most misses to the first-level cache would also miss in the second-level. In this situation, adding a second-level cache can "get in the way" by adding delay between a first-level cache miss and an off-chip access.

An important aspect of this work is the way in which we combine miss-rate data (obtained by simulation) with an on-chip area model and a detailed cache access/cycle time model (CACTI) .

• N.P. Jouppi and S.J.E. Wilton, ``Tradeoffs in Two-Level On-Chip Caching,'' in 21st Annual International Symposium on Computer Architecture , 1994. [also available as DEC WRL technical report number 93/3]. [abstract] [postscript]

Back to Steve Wilton's Home Page