benchmarks evals
The Best Safety Monitor Intervened on Time 40.74% of the Time
PASTABench marks the earliest signal and trigger across 1,139 agent trajectories.
Published Updated Story ID: mp-2026-09-24-015
Summary
PASTABench marks the earliest signal and trigger across 1,139 agent trajectories.
Sixteen models were tested on five risk categories and 13 subcategories. The top result reached 40.74% optimal-timing interventions, while hazard-word removal exposed lexical overfitting in smaller systems.
Why it matters
PASTABench marks the earliest signal and trigger across 1,139 agent trajectories.
Limits and context
No additional limitation was separately recorded.
Key claims
PASTABench marks the earliest signal and trigger across 1,139 agent trajectories.
Evidence: source-2026-09-24-017
Sources
- arXiv preprint 2609.28197arXiv · primary research
Corrections
No corrections have been recorded for this story.