frontier models
Longer Agent Traces Taught Persistence Past the Point of Use
A 12,000-token supervision horizon beat 16,000 tokens on Terminal-Bench while taking less training time.
Summary
A 12,000-token supervision horizon beat 16,000 tokens on Terminal-Bench while taking less training time.
Imitating more of a teacher trajectory does not indefinitely improve a terminal agent. In the reported Terminal-Bench experiments, a 12,000-token horizon solved more tasks than a 16,000-token horizon while requiring 30% less training time. Short horizons encouraged premature stopping, intermediate ones supported error recovery and long ones produced over-persistence. The researchers then refined only continuations likely under a short-horizon warm start. At 16,000 tokens, that selective method increased successful attempts from 110 to 126 and improved results on two additional benchmarks; using half the long-horizon data still reduced training time by 23%. The useful lesson is selection, not sheer transcript length.
Why it matters
A 12,000-token supervision horizon beat 16,000 tokens on Terminal-Bench while taking less training time.
Limits and context
- Imitating more of a teacher trajectory does not indefinitely improve a terminal agent.
- The researchers then refined only continuations likely under a short-horizon warm start.
- The useful lesson is selection, not sheer transcript length.
Key claims
A 12,000-token supervision horizon beat 16,000 tokens on Terminal-Bench while taking less training time.
Qualification: Imitating more of a teacher trajectory does not indefinitely improve a terminal agent.
Evidence: source-2026-10-06-014
Sources
- arXiv preprint 2610.05831arXiv · primary research
Corrections
No corrections have been recorded for this story.