safety security
The Tool Told the Truth and Still Hid the Danger
EpiSEM-Bench removes one safety-critical fact at a time to test whether agents seek the missing evidence before acting.
Summary
EpiSEM-Bench removes one safety-critical fact at a time to test whether agents seek the missing evidence before acting.
A tool response can be entirely truthful yet omit the fact that makes an action unsafe. EpiSEM-Bench turns that problem into 300 matched safe and unsafe task pairs across six domains. It suppresses critical facts, measures whether agents execute prohibited actions, and uses an independent audit tool to see whether they recover the missing evidence before proceeding. Across ten tested models, omission robustness and useful recovery varied substantially. Giving agents access to recovery tools reduced unsafe execution for all ten, but some still failed to request or correctly use the needed facts. This is a benchmark result, not a field guarantee for tool-using agents.
Why it matters
EpiSEM-Bench removes one safety-critical fact at a time to test whether agents seek the missing evidence before acting.
Limits and context
- This is a benchmark result, not a field guarantee for tool-using agents.
Key claims
EpiSEM-Bench removes one safety-critical fact at a time to test whether agents seek the missing evidence before acting.
Qualification: This is a benchmark result, not a field guarantee for tool-using agents.
Evidence: source-2026-10-23-005
Sources
- arXiv preprint 2610.31650, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.