gpuemu for OSS-inference maintainers
A one-line GitHub Action that catches numerical kernel regressions before release — the correctness CI that vLLM, SGLang, TensorRT-LLM, llama.cpp, and MLC-LLM keep rebuilding by hand.
Maintainers of open inference engines field correctness regressions constantly — and the tooling to catch them before release doesn’t exist off the shelf, so teams build it badly, in-house, over and over.
The pain, with real issues
- SGLang #21238 (Mar 2026) — degenerate output with Qwen2.5-Math-7B at
temperature=1.0from the FlashInfer sampling kernel; a regression that re-surfaced after being fixed once. - SGLang #17839 — a PCG CUDA-graph change dropped gpt-oss-120b accuracy from 0.80 to 0.745.
- vLLM #26378 (Oct 2025) — “wrong answer with
torch==2.9and Inductor compilation.” - vLLM #20974 — a GPTQ-Int4 regression between 0.9.x point releases.
- llama.cpp #20052 — multi-GPU layer-split producing garbage past 2048 tokens of context on non-P2P PCIe.
- SGLang #15996 — literally an open issue titled “[CI Infrastructure] Unified output format for regression check.” Maintainers are building this by hand.
Every one of these is a silent numerical wrong-output bug that the standard test suite shipped green.
The workflow gpuemu gives you
- One-line GitHub Action.
gpuemu cifuzzes each op against an fp64 oracle, lints the artifacts, diffs against a baseline, and posts a PR comment (or SARIF). - Catches the regressions above. Adversarial inputs hit the partial-tile, high-temperature, and long-context cases these issues lived in.
- A correctness badge for your README that means something measurable.
Stop rebuilding regression CI by hand. Adopt the gate the whole ecosystem needs.
Frequently Asked Questions
How much setup does the GitHub Action need?
One workflow file. gpuemu ci runs fuzz + lint + baseline-diff and emits a PR comment or SARIF. Point it at your reference scripts and it gates every PR.
Will it flag false positives on legitimate numerical drift?
Tolerances are calibrated per op and dtype from a control population (p95 × 1.5), which raised recall to 82% with zero false positives on controls in our study — it flags real regressions, not normal float noise.