For some unexplainable reason, the major components of the PC the microprocessor, the memory, motherboard data bus, the hard disk drive, and so on all operate at different speeds. One would think that they would all be coordinated to operate together. Well, to a certain extent they do, but by and large they are all developed by different companies who are in competition to develop the fastest, biggest, and best computer component.
The two components that must work together closely and constantly are the CPU (microprocessor) and primary memory (RAM). Unfortunately, RAM is faster than the CPU. It is also the design goal of every PC to have the CPU idle as little as possible. If the CPU requests data from RAM, the data must be located and then transferred over the data bus to the CPU. Regardless of how fast RAM is, the CPU must wait while these actions are carried out. This is where caching comes in.
CACHE ON THE PC
Cache memory is very fast computer memory that is used to hold frequently requested data and instructions. As you will see later, it is a little more complicated than that, but cache exists to hold at the ready data and instructions from a slower device (or a process that requires more time) for a faster device. On today's PCs, you will commonly find cache between RAM and the CPU and perhaps between the hard disk and RAM. A cache is any buffer storage used to improve computer performance by reducing its access times. A cache holds instructions and data likely to be requested by the CPU for its next operation. Caching is used in two ways on the PC:
Cache memory A small and very fast memory storage located between the PC's primary memory (RAM) and its processor. Cache memory holds copies of instructions and data that it gets from RAM to provide high speed access by the processor.
Disk cache To speed up the transfer of data and programs from the hard disk drive to RAM, a section of primary memory or some additional memory placed on the disk controller card is used to hold large blocks of frequently accessed data.
SRAM and Cache Memory
Cache memory is usually a small amount of static random access memory or SRAM (see Chapter 7 for more information on SRAM). SRAM is made up of transistors that don't need to be frequently refreshed (unlike DRAM, which is made up of capacitors and must be constantly refreshed).
SRAM has access speeds of 2ns (nanoseconds) or faster; this is much faster than DRAM, which has access speeds of around 50ns. Data and instructions stored in SRAM based cache memory are transferred to the CPU many times faster than if the data were transferred from the PC's main memory. In case you're wondering why SRAM isn't
also used for primary memory, which could eliminate the need for cache memory all together, there are some very good practical and economic reasons. SRAM costs as much as six times more than DRAM and to store the same amount of data as DRAM would require a lot more space on the motherboard.
Caching in Operation
The CPU operates internally faster than RAM is able to supply data and instructions to it. In turn, RAM operates faster than the hard disk. Caching solves the speed issues between these devices by serving as a buffer between faster devices (the processor or RAM) and slower devices (RAM or the hard disk).
The CPU interacts with RAM through a series of wait states. During a wait state, the CPU pauses to allow a certain number of clock cycles for the data it has requested to be located and transferred from RAM to its registers. If the data is not in RAM already and must be fetched from the hard disk, additional wait states are invoked and the CPU waits even longer for its data. One of the primary purposes of the cache memory is to eliminate the cycles burned in CPU wait states. Eliminating any CPU idleness should make the entire system more productive and efficient.
Locality of Reference
The principle of locality of reference is a design philosophy in computing that is based on the assumption that the next data or instructions to be requested is very likely to be located immediately following the last data or instructions requested by the CPU. Using this principle, caching copies data or instructions just beyond the data requested into the cache memory in anticipation of the CPU asking for it. How successful the caching system is at making its assumptions determines the effectiveness of the caching operation.
As iffy as this may sound, PC caching systems surprisingly get a cache hit about 90 to 95 percent of the time. The cache memory's hit ratio determines its effectiveness. Each time the caching system is correct in anticipating which data or instructions the CPU will want and has it in cache, it is tallied as a hit. The number of hits divided by the total requests for data by the CPU is how the hit ratio is calculated. Of course, if the CPU asks for data that is not in cache, the data must be requested from RAM and a cache miss, a definite caching no no, is tallied.
Saving Trips
If your PC did not have cache memory, all requests for data and instructions by the CPU would be served from RAM. Only the data requested would be supplied, and there would be no anticipation of what the CPU would be asking for next. This would be something like if every time you wanted a cold one, you had to run to the store for just one can, bottle, or cup of your favorite drink. If the CPU is very busy, it could get bogged down in memory requests, just like if you were very thirsty, you would spend all of your time running to and from the store.
Adding cache memory to a system is like adding a refrigerator to your situation. If you were able to purchase a six pack or a case of your favorite drink, it would save you a lot of sneaker wear and tear. Caching anticipates what the CPU may next ask for and copies the equivalent of a case of data or instructions to cache memory. As long as the CPU requests the data stored in cache memory, the whole system speeds up. Since, the caching system guesses correctly about 90 to 95 percent of the time, caching saves a tremendous amount of wait cycles for the CPU.
In order to increase the amount of level 1 (L1) cache on a PC, you have to replace the CPU with a processor that is compatible with the motherboard and chipset that includes additional internal L1 cache. On the other hand, level 2 (L2) cache can be upgraded. L2 cache modules are plugged into special cache module mounts or cache memory expansion sockets located on the motherboard (more on this later).
Internal, External, and Levels of Cache
There are two types of cache memory:
Internal cache Also called primary cache; placed inside the CPU chip
External cache Also called secondary cache; located on the motherboard
As briefly touched upon already, cache is also designated by its level, which is an indication of how close to the CPU it is. Cache is designated into two levels, with the highest level of cache being the closest to the CPU (it is usually a part of the CPU, in fact):
Level 1 (L1) cache Level 1 cache is often referred to interchangeably with internal cache, and rightly so. Ll cache is placed internally on the processor chip and is, of course, the cache memory closest to the CPU.
Level 2 (L2) cache Level 2 cache is normally placed on the motherboard very near the CPU, but because it is further away than Ll cache, it is designated as the second level of cache. Commonly, L2 cache is considered the same as external cache, but L2 cache can also be included on the CPU chip. If there is a level 3 to cache, it is RAM.
Ll and L2 cache, as well as internal and external cache, are not exactly levels in the sense that Ll is higher in ranking than L2. The different levels of cache work together, and data is located in either level based on the rules and policies associated with the caching system more on these later.
In contrast to these definitions of cache memory's placement and levels, older PCs, notably those with 286 or 386 processors, do not include cache memory on the CPU. Any cache memory on these PCs must be located on the motherboard and is designated primary (Ll) cache. Yes, this external cache is Ll cache. Not to worry; this is the exception and it is dying as fast as these PCs.
