The problem: every benchmark says your kernel is “correct”
In short: the industry-standard correctness check for a GPU kernel is a single
torch.allclose on one shape, one dtype, one seed. It is blind to tail-mask leaks,
accumulator-scale bugs, missing normalisation, and online-softmax rescale errors. In a measured
26-op corpus it accepted 9 out of 9 LLM-style buggy kernels as correct.
One line decides what ships
The industry-standard correctness oracle for a GPU kernel is one line:
torch.allclose(my_kernel(x), reference(x), atol=1e-5, rtol=1e-2)One shape. One dtype. One seed. Every modern LLM-kernel benchmark — KernelBench, TritonBench, GEAK, KernelBand, STARK — uses the same oracle inside. Kernels that pass it ship to production.
What it misses
That oracle is blind to entire bug classes that LLM-generated CUDA/Triton code routinely contains:
| Bug class | Example | Why allclose misses it |
|---|---|---|
| Tail-mask leak | softmax forgets to mask the last partial tile | Only fires when H isn't a multiple of BLOCK — H=256 looks fine |
| Accumulator scale | matmul writes acc = instead of acc += | Result happens to match within rtol on the chosen shape |
| Missing normalisation | attention without 1/√D | Saturates softmax differently; one shape looks correct |
| Online-softmax rescale | flash-attention forgets acc *= α after a max update | Only wrong when N > BLOCK_N |
A concrete example: the tail-mask leak
A softmax kernel processes its input in tiles of BLOCK columns. If the kernel
forgets to mask the final, partial tile, it averages in garbage — but only when the column
count isn’t a multiple of BLOCK. Test it on H=256 with
BLOCK=64 and it looks perfect. Run it on H=257 in production and
every row is subtly wrong, shifting attention by roughly 0.31 — enough to
degrade a long-context model without ever crashing.
Because the benchmark picked a friendly shape, the bug never surfaces. The compute is still billed. The result is wrong. Nothing crashes.
What gpuemu does instead
gpuemu replaces “allclose on one shape” with an operator-domain–aware regime: an fp64 reference oracle per dtype, op-schema-aware fuzzing that deliberately hits partial tiles and adversarial values, per-op calibrated tolerances, and static PTX/SASS lint. The validation step runs without a GPU, and every failure replays byte-for-byte from its seed.
In our measured corpus, the standard one-shape oracle accepted 9/9 of these buggy kernels. gpuemu’s regime caught all nine, with zero false positives on fifteen control kernels. See the evidence →