frontier models
The Speech Evaluator Was Mishearing the Rare Words
Evaluation recognizers missed 33 to 67 percent of very rare words even in human recordings used as the control.
Summary
Evaluation recognizers missed 33 to 67 percent of very rare words even in human recordings used as the control.
Adding two recognizer families cut apparent very-rare-word TTS error from roughly 10 to 13 percent down to 3 to 5 percent on words at least one system heard. Yet masking recognizer misses introduced its own bias, and the authors conclude that listening tests remain necessary.
Why it matters
Evaluation recognizers missed 33 to 67 percent of very rare words even in human recordings used as the control.
Limits and context
- Yet masking recognizer misses introduced its own bias, and the authors conclude that listening tests remain necessary.
Key claims
Evaluation recognizers missed 33 to 67 percent of very rare words even in human recordings used as the control.
Qualification: Yet masking recognizer misses introduced its own bias, and the authors conclude that listening tests remain necessary.
Evidence: source-2026-10-23-020
Sources
- arXiv preprint 2610.31470, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.