Performance: the compiler errors¶
Level: 301 · anyone whose benchmark did not run
One line: Most mistakes around benchmarks are refused before anything is measured — the go command checks a benchmark's signature when it loads the test files, testing fails a benchmark whose b.Loop was left early or nested, and a mistyped -gcflags reaches the compiler's own flag parser — and the ones to fear are the quiet ones, the runs that measured nothing and still printed ok.
wrong signature for BenchmarkJoin, must be: func BenchmarkJoin(b *testing.B)¶
A benchmark was written with the test parameter, b *testing.T, out of habit.
Verified output of performance_err_signature_sh.sh — regenerated by tools/run_examples.py, never hand-typed.
$ go mod init example
$ go test -bench=. -run='^$'
# example
./line_test.go:5:1: wrong signature for BenchmarkJoin, must be: func BenchmarkJoin(b *testing.B)
FAIL example [setup failed]
exit status 1
$ go vet .
./line_test.go:5:1: wrong signature for BenchmarkJoin, must be: func BenchmarkJoin(b *testing.B)
exit status 1
The mistake — The parameter type is *testing.T. The go command's test loader (load/test.go ↗) checks every Benchmark* function's signature before it compiles anything, which is why go test reports [setup failed] rather than a compiler error, and why go vet ., which loads the test files the same way, prints the same line and exits 1.
The fix —
A benchmark whose name is malformed rather than its signature — Benchmarkjoin — is a vet finding instead, on the vet page.
benchmark function returned without B.Loop() == false (break or return in loop?)¶
A benchmark measured the first iteration and then left the loop with break.
Verified output of performance_err_loop_sh.sh — regenerated by tools/run_examples.py, never hand-typed.
$ go mod init example
$ go test -bench=. -run='^$'
--- FAIL: BenchmarkBreak
benchmark.go:<line>: benchmark function returned without B.Loop() == false (break or return in loop?)
--- FAIL: BenchmarkNested
benchmark.go:<line>: B.Loop called with timer stopped
--- FAIL: BenchmarkMixed
benchmark.go:<line>: B.Loop called with timer stopped
FAIL
exit status 1
FAIL example
exit status 1
The mistake — BenchmarkBreak leaves for b.Loop() before Loop has returned false. Loop manages the timer and the iteration count, and a loop that ends any other way leaves it mid-measurement; testing fails the benchmark instead of reporting a number from one iteration. The benchmark.go:<line> is a line in the testing package's source, replaced here because it moves between Go versions.
The fix — let the loop run to the end, and put the decision outside it:
B.Loop called with timer stopped¶
One b.Loop loop inside another, and a b.Loop loop inside a b.N loop.
The mistake — BenchmarkNested and BenchmarkMixed, in the transcript above. When the inner Loop returns false it stops the timer; the outer loop's next call to Loop then finds the timer stopped and fails the benchmark. Loop's documentation is explicit that a benchmark uses either Loop or a loop to b.N, never both, and that b.Loop() must be the loop condition itself, written exactly so.
The fix — one loop, and any sub-cases as sub-benchmarks:
func BenchmarkJoin(b *testing.B) {
for _, n := range []int{2, 8} {
b.Run(fmt.Sprint(n), func(b *testing.B) {
for b.Loop() {
strings.Join(fields[:n], ";")
}
})
}
}
flag provided but not defined: -mm¶
-gcflags=-mm was meant to be the verbose inlining report.
Verified output of performance_err_gcflags_sh.sh — regenerated by tools/run_examples.py, never hand-typed.
$ go mod init example
$ go build -gcflags=-mm .
# example
flag provided but not defined: -mm
usage: compile [options] file.go...
exit status 1
$ go build -gcflags=-d=ssa/check_bec .
# example
compile: Did not find a phase matching check_bec in -d=ssa/... debug option
exit status 1
$ go build -gcflags=-d=ssa/check_bce .
exit status 0
The mistake — -gcflags hands its value to the compiler, compile, whose flag parser does not know -mm; it prints its usage (cut to one line here) and the build fails. The verbose form is -m=2 or -m -m.
The fix —
Did not find a phase matching check_bec in -d=ssa/... debug option¶
A typo in the SSA debug flag from Bounds checks, and how to drop them.
The mistake — the second command in the transcript above. -d=ssa/<phase>/<flag> names a compiler pass, and the compiler looks the name up before running; check_bec is not a pass. The third command shows the spelling that works, and, on a program with nothing to check, that a clean run prints nothing at all.
The fix —
[no test files], and PASS with nothing measured¶
A go test -bench that found nothing to run, and said so quietly.
Verified output of performance_err_nobench_sh.sh — regenerated by tools/run_examples.py, never hand-typed.
$ go mod init example
$ go test -bench=. -run='^$'
? example [no test files]
exit status 0
$ go test -bench=. -run='^$' # line_test.go added: a test, no benchmark
PASS
ok example
exit status 0
$ go test -bench='Join(' -run='^$'
testing: invalid regexp for element 0 of -test.bench ("Join("): error parsing regexp: missing closing ): `Join(`
exit status 1
FAIL example
exit status 1
$ go test -bench=. -benchtime=fast -run='^$'
invalid value "fast" for flag -test.benchtime: invalid duration
exit status 1
The mistake — The first run is in a package with no _test.go at all: ? example [no test files], exit status 0. The second has a test file and a -bench=. that matches no Benchmark* function: PASS, ok, exit status 0, and no benchmark line — the run measured nothing and a CI step that only checks the exit status is satisfied. Neither is an error to the go command.
The fix — look for a benchmark line in the output, or run go test -list 'Benchmark.*' first to see what -bench will match:
invalid regexp for element 0 of -test.bench¶
The third command in the transcript above: -bench='Join(' was meant to match BenchmarkJoin.
The mistake — -bench takes a regular expression, not a name, and Join( has an unclosed group. The test binary reports it and exits 1, which go test turns into FAIL.
The fix —
invalid value "fast" for flag -test.benchtime: invalid duration¶
The last command in the transcript above.
The mistake — -benchtime takes a duration such as 2s or an iteration count such as 2000x. Anything else fails the flag parser before a benchmark runs; the usage text that follows the line is cut here.
The fix —