robotics
Coding Agents Build the Robot Worlds—and Still Fail Every Test
RoboWorld contains 25 generated simulations and 25 test tasks; the best frontier models averaged 0.06 reward and solved none of them.
Summary
RoboWorld contains 25 generated simulations and 25 test tasks; the best frontier models averaged 0.06 reward and solved none of them.
Gym-Anything for Robotics treats simulation construction as a coding-agent task. Its adversarial creation-and-audit loop checks simulations and tasks chosen to cover economically valuable physical work. The resulting RoboWorld set contains 25 simulations, 25 test environments and 175 training environments. Even the best evaluated frontier models failed every test task, averaging 0.06 reward. Repeated attempts, privileged state-reading teachers, self-assisted learning and code-generated demonstrations each improved results. Because the worlds are agent-built and audited rather than measurements of physical workplaces, the benchmark tests learning inside the released simulations, not real-world robot competence.
Why it matters
RoboWorld contains 25 generated simulations and 25 test tasks; the best frontier models averaged 0.06 reward and solved none of them.
Limits and context
- Because the worlds are agent-built and audited rather than measurements of physical workplaces, the benchmark tests learning inside the released simulations, not real-world robot competence.
Key claims
RoboWorld contains 25 generated simulations and 25 test tasks; the best frontier models averaged 0.06 reward and solved none of them.
Qualification: Because the worlds are agent-built and audited rather than measurements of physical workplaces, the benchmark tests learning inside the released simulations, not real-world robot competence.
Evidence: source-2026-10-30-009
Sources
- arXiv preprint 2610.40565, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.