benchmarks evals
The Right Answer Was Missing From the Menu
A paired benchmark measures rejection separately from classification accuracy.
Summary
A paired benchmark measures rejection separately from classification accuracy.
The candidate-coverage study tests Laya and Jev on matched requests with a reference option present or absent. Native rejection and false-rejection rates differed sharply; calibration changed the tradeoff. The evaluation covers reference-label omission and explicitly leaves natural out-of-scope generalization unresolved.
Why it matters
A paired benchmark measures rejection separately from classification accuracy.
Limits and context
No additional limitation was separately recorded.
Key claims
A paired benchmark measures rejection separately from classification accuracy.
Evidence: source-2026-10-04-020
Sources
- arXiv preprint 2610.03387, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: This October 4 edition was reconstructed on October 5 after the workstation was offline. It uses original October 2 submissions from the standard Sunday-evening arXiv release, rather than claiming those papers were available for the missed 9 AM run. The date and 9 AM publication slot preserve archive chronology; October 5 developments are excluded. The September 27 Invention Desk is carried over because its replacement was generated October 5.