The optimizer deletes your benchmark¶

Level: 301 · deep dive

One line: Time a loop whose result nobody uses and an optimizing compiler removes the loop; keep the result with a DoNotOptimize barrier and Clang may still replace the loop with a formula — the barrier protects a value, not the work that produced it.

#include <cstdint>

// The barrier Google Benchmark's DoNotOptimize is built on — GCC and Clang only.
template <class T>
void do_not_optimize(T& value) {
#if defined(__clang__)
    asm volatile("" : "+r,m"(value) : : "memory");
#else
    asm volatile("" : "+m,r"(value) : : "memory");
#endif
}

std::uint64_t sum_below(std::uint64_t n) {
    std::uint64_t total = 0;
    for (std::uint64_t i = 0; i < n; ++i) {
        total += i;
        do_not_optimize(total);   // every step's result must exist, so every step runs
    }
    return total;
}

A barrier inside the loop keeps every step; a barrier on the result keeps only the result. For real work, use Google Benchmark ↗, which does this and the repetition and statistics besides — the talk's own advice on writing your own harness is don't.

Four ways to time one loop¶

The example sums the numbers below 100,000,000 four ways, and times each with steady_clock:

Variant	What keeps the work alive
result thrown away	nothing
result stored to a `volatile`	the store has to happen
result passed to `do_not_optimize`	the value has to exist at that point
`do_not_optimize` on every step	every partial sum has to exist

Verified output of the_optimizer_deletes_your_benchmark.cpp — regenerated by tools/run_examples.py and held to the same answer key on libstdc++ and libc++ in CI, never hand-typed.

result stored to a volatile  = 4999999950000000
result passed to the barrier = 4999999950000000
barrier on every step        = 4999999950000000

That is the answer key, and all it can hold is the sums: every compiler must produce the same number. How long each version took is the part no two compilers agree on. Run it with --times to see yours:

c++ -std=c++20 -O2 examples/the_optimizer_deletes_your_benchmark.cpp -o bench && ./bench --times

Measured 2026-09-10 22:25 EDT — Intel Core i5-10500, macOS 26.6.2, load average ~6 — three runs each, range — summary — not an answer key

                               Apple clang 21     GCC 15.2 (Homebrew)    GCC 13.4 (Docker,
                               libc++, macOS      libstdc++, macOS       Linux VM)
result thrown away             73–91 ns           0 ns (1 µs clock)      77–84 ns
result stored to a volatile    42–63 ns           36.1–36.4 ms           37.4–37.7 ms
result passed to the barrier   42–61 ns           36.6–36.7 ms           37.2–37.7 ms
barrier on every step          135.5–137.3 ms     24.3–29.9 ms           24.9–25.7 ms

Read it by rows:

Thrown away — every compiler deleted the loop; what is left is the cost of reading the clock twice.
Volatile, and barrier on the result — GCC ran a loop, about 37 ms. Clang did not: 42–63 ns is the price of two clock reads and nothing else. It still produced the right answer — the key proves that — so it computed it some other way.
Barrier on every step — both ran all 100 million iterations, and Clang took five times as long as GCC, for a reason visible in its assembly.

Where Clang's loop went¶

Apple clang 21, x86-64 macOS — clang++ -std=c++20 -O2 -S — sum_below, with .cfi and .p2align directives removed

__Z9sum_belowy:                         ## @_Z9sum_belowy
## %bb.0:
    pushq   %rbp
    movq    %rsp, %rbp
    testq   %rdi, %rdi
    je  LBB0_1
## %bb.2:
    leaq    -1(%rdi), %rax
    leaq    -2(%rdi), %rcx
    mulq    %rcx
    shldq   $63, %rax, %rdx
    leaq    (%rdi,%rdx), %rax
    decq    %rax
    popq    %rbp
    retq
LBB0_1:
    xorl    %eax, %eax
    popq    %rbp
    retq

No backward jump, so no loop. mulq multiplies n−1 by n−2 into a 128-bit result, and the shift and adds turn it into the closed form of the sum, n(n−1)/2. Clang recognised the loop as an arithmetic series and replaced it; the barrier asked for the result, and got it. The same happens on arm64 — the architecture of the macOS CI runner: Clang 19.1 for AArch64 on Compiler Explorer compiles sum_below to mul, umulh and extr, and no loop.

GCC's loop¶

GCC 13.4, x86-64 Linux — g++ -std=c++20 -O2 -S — sum_below, with .cfi and .p2align directives removed

_Z9sum_belowm:
.LFB5296:
    testq   %rdi, %rdi
    je  .L4
    xorl    %eax, %eax
    xorl    %edx, %edx
    testb   $1, %dil
    je  .L3
    movl    $1, %eax
    cmpq    $1, %rdi
    je  .L1
.L3:
    leaq    1(%rdx,%rax,2), %rdx
    addq    $2, %rax
    cmpq    %rax, %rdi
    jne .L3
.L1:
    movq    %rdx, %rax
    ret
.L4:
    xorl    %edx, %edx
    movq    %rdx, %rax
    ret

A real loop — jne .L3 jumps back — unrolled by two: each trip adds i and i + 1 with one leaq. Both compilers are right. The as-if rule lets a compiler emit any program with the same observable behaviour, and how long a program takes is not observable behaviour (intro.abstract ↗). Why GCC's two-at-a-time loop is slower here than its one-at-a-time loop below is a question about the CPU, not the compiler — one of the chapter's outlines.

The barrier costs something too¶

Clang's loop for barrier on every step:

Apple clang 21, x86-64 macOS — the same -S output — the loop in main, from its label to its backward jump

LBB1_12:                                ## =>This Inner Loop Header: Depth=1
    addq    %rax, -48(%rbp)
    ## InlineAsm Start
    ## InlineAsm End
    incq    %rax
    cmpq    -56(%rbp), %rax
    jb  LBB1_12

GCC's:

GCC 13.4, x86-64 Linux — the same -S output — the loop in main, from its label to its backward jump

.L78:
    addq    %rdx, %rbx
    addq    $1, %rdx
    cmpq    %rdx, %r14
    jne .L78

"+r,m" lets the compiler keep value in a register or in memory, and Clang chose memory: total lives at -48(%rbp) and is read, added to and written back on every iteration, and n is re-read from -56(%rbp) each time, because the "memory" clobber says any memory may have changed. GCC kept both in registers. The barrier that made the loop run also changed what the loop costs — fivefold here — which is why a barrier belongs on a benchmark's inputs and outputs rather than inside its hot loop, wherever the benchmark allows it.

Why `volatile` is not the fix¶

volatile std::uint64_t sink = sum_below(n); does exactly what it says: the store to sink happens. Volatile accesses are part of a program's observable behaviour (intro.abstract ↗) — but the value being stored is not, and nothing says how it was computed. Clang stored the formula's answer. The talk adds the other half from JF Bastien's P1152R0 ↗: volatile operations keep their order relative to each other, but "may change relative to non-volatile operations" — and reading the clock is not a volatile operation.

Work can leave the timed region¶

Even when the work survives, nothing says when it happens. The talk's A bigger problem? slide asks whether a compiler may compute the result before the first now() and leave the timed region empty — and it may, whenever it can prove the difference is not observable. Mike Spertus proposed a timing_fence() in P0342R0 ↗ in 2016, and the talk records it as rejected: a fence inside a now() compiled in another file is invisible to the compiler compiling yours. Google Benchmark's answer is a second primitive beside DoNotOptimize, ClobberMemory(), which in its current source is std::atomic_signal_fence(std::memory_order_acq_rel) — a standard, compiler-only fence rather than an asm statement (utils.h ↗).

Google Benchmark today¶

The slide builds DoNotOptimize(const T&), whose constraint "r,m" only reads the value. Google Benchmark still ships that overload and now marks it deprecated — "DoNotOptimize(T const&) can permit undesired compiler optimizations" — asking for a non-const lvalue instead (utils.h ↗). The overload it recommends is the one this page uses: it takes T& and declares that the value may be written, so the compiler cannot assume it knows the value afterwards. It spells the constraint differently for the two compilers — "+r,m" for Clang, "+m,r" for GCC — and the example copies that split for a measured reason: given "+r,m", GCC 14.4 and 15.2 stop with "impossible constraint in 'asm'". MSVC has no inline assembly on x64 at all.

Even the recommended form protects the value, not the computation: the barrier on the result row above is exactly that form.

The standardisation gap¶

C++ has no standard way to say do not optimize this away. Mikhail Maltsev's P0412R0 ↗ (2016) proposed keep() and touch(), and in the talk's words it "Stalled at R0". Rust has shipped std::hint::black_box ↗ since 1.66 and Zig has mem.doNotOptimizeAway; the slide's last line reads "C++: …".

If you are coming from another language¶

Rust. std::hint::black_box is this barrier in the standard library, promised only on a "best-effort" basis. rustc is built on LLVM, like Clang, and turns the same result-only loop into the same formula: black_box is a hint ↗, and constant folding in What the optimizer does ↗.
Python. CPython's compiler folds constant expressions but does not delete a loop whose result is unused, so a Python timing does measure the loop — together with the interpreter's own overhead, which is the trap there. timeit ↗ repeats the measurement for you and turns garbage collection off while it runs.