safety security
Retrying a “Safe” Proposal Multiplied Its Risk
Per-candidate certification can inflate false admission under best-of-k selection; the paper derives a setwise condition that survives generator replacement.
Summary
Per-candidate certification can inflate false admission under best-of-k selection; the paper derives a setwise condition that survives generator replacement.
The analysis asks when a runtime safety gate can remain trustworthy even if the learned policy or language-model planner behind it changes. Certifying each candidate separately does not compose: with k retries, a false-admission rate alpha can grow to one minus one-minus-alpha raised to k. The authors prove that simultaneous setwise soundness is necessary and sufficient for generator-independent admission soundness, provided there is a design-time certificate and no bypass path. A second result formalizes what partial observation makes impossible when different hidden states demand different safe actions. The contribution is a theoretical contract and risk ledger, not evidence from a deployed controller.
Why it matters
Per-candidate certification can inflate false admission under best-of-k selection; the paper derives a setwise condition that survives generator replacement.
Limits and context
- The analysis asks when a runtime safety gate can remain trustworthy even if the learned policy or language-model planner behind it changes.
- Certifying each candidate separately does not compose: with k retries, a false-admission rate alpha can grow to one minus one-minus-alpha raised to k.
- The contribution is a theoretical contract and risk ledger, not evidence from a deployed controller.
Key claims
Per-candidate certification can inflate false admission under best-of-k selection; the paper derives a setwise condition that survives generator replacement.
Qualification: The analysis asks when a runtime safety gate can remain trustworthy even if the learned policy or language-model planner behind it changes.
Evidence: source-2026-09-09-003
Sources
- arXiv preprint 2609.06036arXiv · primary research
Corrections
No corrections have been recorded for this story.