The Loose Test Passed. The Contract Found 62 Percent Broken

A loose kernel test can compare a few random outputs at one shape and still miss nondeterminism, shape failures, NaN and infinity handling, or low-precision accumulation. The authors built a twelve-gate verifier, including several tolerance-free checks, and applied it to 2,638 machine-generated kernels already accepted by a public system. Their audit found 39.5 percent broken beyond any tolerance dispute and 62.1 percent carrying at least one violation; the standard test accepted 1,487 kernels that the stricter verifier rejected.

The paper backs the audit with positive controls, a tolerance sweep, comparison to the reference benchmark's own correctness code and a stratified hand review. The same verifier also checked the authors' own native Blackwell training-backward kernel for the gated-linear-recurrence family against a double-precision oracle. The result is a preprint audit of one corpus and one contract design, not proof that every generated kernel is unsafe, but it sharply weakens headline correctness rates built on a single permissive test.