gpuemu vs framework fuzzers (FreeFuzz, DocTer, NablaFuzz, FuzzGPT)
API-level deep-learning fuzzers test the framework's Python API surface. gpuemu tests the kernel. The distinction matters: an ACL TOSEM 2025 study measured these fuzzers at a 6.5% real-world bug catch rate.
| Capability | gpuemu | FreeFuzz / DocTer / NablaFuzz / FuzzGPT |
|---|---|---|
| Targets the kernel (not just the API) | Yes | No |
| fp64 reference oracle | Yes | Partial |
| Op-schema-aware shape generation | Yes | Partial |
| Per-op calibrated tolerances | Yes | No |
| Byte-for-byte failure replay | Yes | No |
FreeFuzz, DocTer, NablaFuzz, and FuzzGPT are research-grade fuzzers for deep-learning frameworks. They generate API calls — argument combinations, unusual dtypes, gradient checks — and look for crashes or inconsistencies.
That’s a different target. They test the framework’s Python surface; gpuemu tests whether a specific kernel computes the reference answer. An ACL TOSEM 2025 evaluation measured the real-world bug-catch rate of this family at 6.5% (34 of 517) — useful, but not a correctness gate for the kernels you ship.
gpuemu works one layer down: a high-precision fp64 oracle, op-schema-aware adversarial inputs tuned to each operator, calibrated tolerances, and replayable failures.
Frequently Asked Questions
Aren't API fuzzers already finding kernel bugs?
They find some, but at the API layer — and an ACL TOSEM 2025 study measured their real-world bug catch at 6.5% (34 of 517 reported). gpuemu works one layer down, comparing kernel output to a high-precision reference on adversarial inputs.