developer tools
The Tests Passed One Reference and Rejected Valid Alternatives
TestPrism evaluates generated tests against multiple correct and incorrect implementations.
Summary
TestPrism evaluates generated tests against multiple correct and incorrect implementations.
Across 300 tasks and 3,000 candidate implementations, joint success required failing the initial program, accepting every valid candidate and rejecting every invalid one. Baseline configurations reached 28% on that measure versus 59.67% for single-reference success. TestHelix improved joint success by 8.67 to 9 percentage points in its two-model comparison.
Why it matters
TestPrism evaluates generated tests against multiple correct and incorrect implementations.
Limits and context
No additional limitation was separately recorded.
Key claims
TestPrism evaluates generated tests against multiple correct and incorrect implementations.
Evidence: source-2026-10-09-020
Sources
- arXiv preprint 2610.12289, original v1arXiv / submitting research authors · primary research
Corrections
No corrections have been recorded for this story.