benchmarks evals
Seven Hundred Twenty-Five Agent Errors Recovered Without Intervention
ReInBench records 1,403 error events across 1,259 computer-use and web traces; 725 recovered, often within a few further steps.
Summary
ReInBench records 1,403 error events across 1,259 computer-use and web traces; 725 recovered, often within a few further steps.
An external monitor that interrupts every agent error can erase the opportunity for the agent to repair its own execution. ReInBench collects 1,259 traces and 1,403 observed error events from computer-use and web tasks. Of those events, 725 self-recovered, often after only a few additional steps, with recovery rates varying across execution models. Recovery from one error also correlated with handling later errors of the same type and with eventual task completion. The authors propose ReCoF to predict at error time whether recovery is likely, reporting a 9.02 percent relative Macro-F1 improvement over the strongest baseline. The result supports selective intervention on this benchmark, not unrestricted waiting in high-consequence systems.
Why it matters
ReInBench records 1,403 error events across 1,259 computer-use and web traces; 725 recovered, often within a few further steps.
Limits and context
- Of those events, 725 self-recovered, often after only a few additional steps, with recovery rates varying across execution models.
- The result supports selective intervention on this benchmark, not unrestricted waiting in high-consequence systems.
Key claims
ReInBench records 1,403 error events across 1,259 computer-use and web traces; 725 recovered, often within a few further steps.
Qualification: Of those events, 725 self-recovered, often after only a few additional steps, with recovery rates varying across execution models.
Evidence: source-2026-10-28-004
Sources
- arXiv preprint 2610.36471, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.