CPUs start executing instructions by fetching those instruction bytes from memory and decoding them into internal operations (micro-ops). Getting data from memory and operating on it consumes power and incurs latency. Micro-op caching is a popular technique to improve on both fronts, and involves caching micro-ops that correspond to frequently executed instructions.
AMD’s recent CPUs have particularly large micro-op caches, or op caches for short. Zen 4’s op cache can hold 6.75K micro-ops, and has the highest capacity of any op cache across the past few generations of CPUs. This huge op cache enjoys high hitrates, and gives the feeling AMD is leaning harder on micro-op caching than Intel or Arm. That begs the question of how the core would handle if its big, beautiful op cache stepped out for lunch.
↫ Chester Lam at Chips and Cheese
The results of turning off the op cache were far less dramatic than one would expect, and this mostly comes down to the processor having to wait on other bottlenecks anyway, like the memory, and a lot of tasks consisting of multiple types of operations which not all make use of op cache. While it definitely contributes to making Zen 4 cores faster overall, even without it, it’s still an amazing core that outperforms its Intel competition.
As a sidenote, this is such a fun and weird thing to do and benchmark. It doesn’t serve much of a purpose, and the information gained isn’t very practical, but turning off specific parts of a processor and observing the consequences does create some insight into exactly how a modern processor works. There are so many different elements that make up a modern processor now, and just gigahertz or even the number of cores barely tells even half the story.
Anyway, we need more of these weird benchmarks.