safety security
The Confidence Gate Failed Outside Finance
Selective abstention cut in-domain sentence error below 2 percent, but found no useful clean subset under extreme social-media shift.
Summary
Selective abstention cut in-domain sentence error below 2 percent, but found no useful clean subset under extreme social-media shift.
The study stress-tested financial named-entity recognizers across SEC filings, financial news and general social media. Whole-output probability was the best in-domain error signal but deteriorated under shift; span probability and self-consistency were more robust. Abstention reduced sentence error from 34.3 percent to below 2 percent on the most confident 40 percent of in-domain inputs and remained useful on news, but failed to recover a usefully large clean subset on the extreme out-of-domain tier. The result favors upstream shift detection before confidence gating.
Why it matters
Selective abstention cut in-domain sentence error below 2 percent, but found no useful clean subset under extreme social-media shift.
Limits and context
No additional limitation was separately recorded.
Key claims
Selective abstention cut in-domain sentence error below 2 percent, but found no useful clean subset under extreme social-media shift.
Evidence: source-2026-08-22-016
Sources
- arXiv preprint 2608.19558arXiv · primary research
Corrections
No corrections have been recorded for this story.