developer tools
The Harness Changed the Solver
Keeping model weights fixed, context compaction and stall handling changed coding-agent results.
Summary
Keeping model weights fixed, context compaction and stall handling changed coding-agent results.
On a 169-task tight-context SWE-bench Verified cohort, the treatment raised complete solutions from 43 to 72 and improved mean fail-to-pass fraction. Wider-context results were closer, reinforcing that evaluations measure a model-harness pair rather than model weights alone.
Why it matters
Keeping model weights fixed, context compaction and stall handling changed coding-agent results.
Limits and context
No additional limitation was separately recorded.
Key claims
Keeping model weights fixed, context compaction and stall handling changed coding-agent results.
Evidence: source-2026-08-30-019
Sources
- arXiv preprint 2608.26218arXiv · primary research
Corrections
No corrections have been recorded for this story.