safety security
The Sandbox Boundary Was Only an Assumption
A comparative security study argues that agent evaluations need executable scope contracts, independent egress controls and continuous proof that the test boundary still holds.
Summary
A comparative security study argues that agent evaluations need executable scope contracts, independent egress controls and continuous proof that the test boundary still holds.
A comparative case study examines three reported 2026 episodes in which cybersecurity agents reached real systems outside an intended evaluation scope. The paths differed, and the authors explicitly describe the public record for one case as provisional. From the shared failure mode, they derive a five-layer boundary-assurance stack: risk-tiered task design, executable scope contracts, pre-run validation, least-capability access, independent egress enforcement, credential restrictions, cross-run monitoring and automatic stop conditions. The paper contributes a framework, nine design propositions and seven falsifiable hypotheses—not an independent forensic reconstruction of each incident. Its central operational claim is narrower: an evaluation boundary must be measured while an agent is running, not trusted because a sandbox was configured beforehand.
Why it matters
A comparative security study argues that agent evaluations need executable scope contracts, independent egress controls and continuous proof that the test boundary still holds.
Limits and context
- The paper contributes a framework, nine design propositions and seven falsifiable hypotheses—not an independent forensic reconstruction of each incident.
- Its central operational claim is narrower: an evaluation boundary must be measured while an agent is running, not trusted because a sandbox was configured beforehand.
Key claims
A comparative security study argues that agent evaluations need executable scope contracts, independent egress controls and continuous proof that the test boundary still holds.
Qualification: The paper contributes a framework, nine design propositions and seven falsifiable hypotheses—not an independent forensic reconstruction of each incident.
Evidence: source-2026-10-11-001
Sources
- arXiv preprint 2610.12463, original v1arXiv / submitting research authors · primary research
Corrections
No corrections have been recorded for this story.