The bug that doesn't crash

A crash is a gift. It tells you where, it tells you when, and it stops the wrong answer from going anywhere. The bugs that cost real money are the ones that do none of that. The code runs to completion, the output has the right shape and a plausible range, and somewhere in it a number is off by a little. Nobody notices until something downstream behaves strangely weeks later.

These are the bugs I write benchmark tasks around, because they are where models, and plenty of engineers, slip. This post is about the pattern, with no client task content in it.

The canonical case: a quantised kernel

Quantisation stores numbers in fewer bits, int8 instead of float32, to make models smaller and faster. The arithmetic needs a scale factor and sometimes a zero point to map between the two ranges. Get the scale slightly wrong, accumulate in a type that overflows, or round in the wrong direction, and the kernel still runs. It returns numbers that are near the right ones. On a single test input the difference might be within noise. Across a model it degrades accuracy by a few points and nobody can say why.

A task built on this pattern is hard for the right reasons. The model has to understand what the code is supposed to compute, construct inputs that expose the drift, and reason about numerics rather than pattern-match a stack trace. There is no error message to search for.

Other bugs in the same family

• Off-by-one in a window or a boundary. The loop runs, one element is dropped or doubled, aggregates shift slightly.
• Timezone and daylight-saving handling. Dates parse, totals are right eleven months a year.
• Floating-point accumulation order. Parallel sums differ from serial ones; a test with a tight equality check flakes, a loose one hides the bug.
• Unit mismatches. Milliseconds in, seconds assumed. Everything runs; everything is a thousand times off in one branch.
• Random seed not propagated. Results are reproducible on the author's machine and nowhere else.

How to test for a bug that does not crash

1. 1.Compare against a reference, not against expected output. A slow, obviously correct implementation computes the truth; the test checks the fast path against it.
2. 2.Use a justified tolerance. Not 1e-9 because it is small, and not 0.1 because the test passes. The tolerance is derived from the numerics and written in a comment.
3. 3.Probe the edges. Maximum and minimum representable values, zero, negative zero, lengths of one and of a power of two, inputs that force rounding.
4. 4.Check a statistic across many inputs. Mean absolute error across a thousand random inputs catches drift that one sample hides.
5. 5.Make the hidden tests different from the visible ones. If the agent can see the inputs the test uses, it can special-case them.

Why this matters for benchmarks

A benchmark full of crash-and-fix tasks measures whether a model can read a stack trace. A benchmark with silent-bug tasks measures whether it can reason about what code is for. The second is the capability labs are paying to measure, and it is the kind of task that gets accepted at review because it cannot be gamed by pattern matching.