benchmarks evals
Put More Tasks in the Prompt, and Each One Gets Less Thought
Errors under composition tracked an implicit per-task reasoning budget even when the full prompt fit comfortably in context.
Summary
Errors under composition tracked an implicit per-task reasoning budget even when the full prompt fit comfortably in context.
Models that solve tasks separately can fail when those same tasks are bundled. Across open-weight systems and Claude Opus 5, the authors found degradation beyond independent error accumulation: models used fewer tokens per task under composition. Problems needing longer traces alone were more fragile when grouped. Sorting tasks by fragility manipulated that allocation, cutting token use by 43 percent across tested math and language tasks while preserving accuracy. The mechanism is inferred from behavioral measurements and may not transfer to every model or prompting setup.
Why it matters
Errors under composition tracked an implicit per-task reasoning budget even when the full prompt fit comfortably in context.
Limits and context
- The mechanism is inferred from behavioral measurements and may not transfer to every model or prompting setup.
Key claims
Errors under composition tracked an implicit per-task reasoning budget even when the full prompt fit comfortably in context.
Qualification: The mechanism is inferred from behavioral measurements and may not transfer to every model or prompting setup.
Evidence: source-2026-10-21-016
Sources
- arXiv preprint 2610.27190, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.