Advanced techniques for cache replacement policies

Modern processors increasingly rely on instruction-level parallelism to enhance execution speed. With the continuous miniaturization of silicon technology, it has become possible to design cores with very wide issue units; however, the degree of parallelism attainable through wider issue widths varies significantly across workloads and is ultimately constrained by inherent instruction dependencies. This work tries to explore methods of detecting which instructions act as the bottleneck for execution, and then presents some possible strategies to prioritize these critical instruction over others along a CPU's pipeline. The focus is on load instruction, as they typically are the highest latency instructions and offer multiple opportunities for optimization along the memory system. The main idea centers on cache replacement policies. Traditional approaches have largely relied on reuse estimation as the only metric. This study explores the possibility of incorporating load criticality into the policy, with the expectation that misses on critical loads will be more expensive than misses on non-critical loads. The study first introduces the concept of load criticality and reviews existing academic approaches to predicting critical loads. Based on these insights, several predictor implementations are designed and evaluated. An optimization of the cache replacement policy is also proposed and analyzed, highlighting its role in reducing memory-related stalls. Finally, a set of optimizations within the CPU pipeline is examined, and their impact on overall performance is analyzed.