safety
Confidence Was Not a Reason to Rely on the Recommendation
A four-tier certificate measures how stable a model's preference is and how broadly it survives contextual changes when ground truth is absent.
Summary
A four-tier certificate measures how stable a model's preference is and how broadly it survives contextual changes when ground truth is absent.
The proposed epistemic-warrant ladder classifies pairwise recommendations as unstable, context-dependent, locally supported or broadly supported. Known-groups tests recovered expert-prespecified orderings, and stronger certificates aligned with independent crowd consensus. Verbalized confidence did not explain the same information, nor could decision difficulty readily account for it. The contribution is a decision-level reliance framework validated in the authors' experiments, not a guarantee that a warranted answer is objectively true.
Why it matters
A four-tier certificate measures how stable a model's preference is and how broadly it survives contextual changes when ground truth is absent.
Limits and context
- Verbalized confidence did not explain the same information, nor could decision difficulty readily account for it.
- The contribution is a decision-level reliance framework validated in the authors' experiments, not a guarantee that a warranted answer is objectively true.
Key claims
A four-tier certificate measures how stable a model's preference is and how broadly it survives contextual changes when ground truth is absent.
Qualification: Verbalized confidence did not explain the same information, nor could decision difficulty readily account for it.
Evidence: source-2026-09-04-012
Sources
- arXiv preprint 2609.04127arXiv · primary research
Corrections
No corrections have been recorded for this story.