benchmarks evals
The Agent Couldn’t Move Until Its Twin Replayed the Past
An executable world model cleared 179 of 183 hidden-rule levels after every prediction mismatch became a counterexample for repair.
Summary
An executable world model cleared 179 of 183 hidden-rule levels after every prediction mismatch became a counterexample for repair.
Twin made a coding agent earn each real action by first reproducing every transition it had already observed inside an executable test-time world model. When the model predicted a result that the environment contradicted, the mismatch became a counterexample and the agent repaired the twin before continuing. Across the reported ARC-AGI-3 runs, the system cleared 179 of 183 levels and inferred the goal before receiving a reward on 156 of the levels it cleared.
The same base model scored 7.8 percent when playing directly, 61.1 with an off-the-shelf harness and 93.3 with the twin-world harness across 25 games. Those figures come from one benchmark family whose grid-game prior is unusually amenable to executable modeling; they do not establish a general recipe for unconstrained physical environments. The sharper result is procedural: prediction errors were not merely logged—they blocked action until the accumulated history could be replayed.
Why it matters
An executable world model cleared 179 of 183 hidden-rule levels after every prediction mismatch became a counterexample for repair.
Limits and context
- Those figures come from one benchmark family whose grid-game prior is unusually amenable to executable modeling; they do not establish a general recipe for unconstrained physical environments.
- The sharper result is procedural: prediction errors were not merely logged—they blocked action until the accumulated history could be replayed.
Key claims
An executable world model cleared 179 of 183 hidden-rule levels after every prediction mismatch became a counterexample for repair.
Qualification: Those figures come from one benchmark family whose grid-game prior is unusually amenable to executable modeling; they do not establish a general recipe for unconstrained physical environments.
Evidence: source-2026-08-17-001
Sources
- arXiv preprint 2608.14490arXiv · primary research
Corrections
No corrections have been recorded for this story.