The cache changes the answer

Level: 301 · deep dive

Stub — an outline, not a lesson. There is no runnable example behind this page yet, so nothing on it has been through the check that backs every other claim in this library. The bullets below are the questions the finished page has to answer.

One line: The same code on the same machine runs at several different speeds depending on whether its data fits in L1, in L3, or only in main memory — so a benchmark's input size is part of its answer.

The questions this page has to answer

• Same code, different sizes: the talk's matrix multiply at 32×32 fits in L1, at 512×512 in L3, and at 2048×2048 spills to memory. What does each cost per element on this machine?
• Loop order: ijk against ikj. The talk says they are 5–10× apart and that the compiler vectorises only the stride-1 one — is that true on both CI compilers?
• Why do the crossover points move from machine to machine, so that a benchmark is true only where it ran?
• How the measurement disturbs what it measures: interrupts, context switches, and frequency scaling, which runs the first iterations on a slower CPU than the warm ones.
• Hot cache or cold — which one is the question you are asking?

In the talk

Caching ruins everything, which the published deck places after the conclusion, with some of its numbers still marked TODO in the source (slide source ↗).

See also

• The branch predictor learns your benchmark
• One number is not a measurement