robotics
Three Agents Build the Room Before the Robot Enters
SceneSmith coordinates designer, critic, and orchestrator agents to create object-rich 3D rooms that load directly into physics simulators for robot training.
Summary
SceneSmith coordinates designer, critic, and orchestrator agents to create object-rich 3D rooms that load directly into physics simulators for robot training.
MIT CSAIL and Toyota Research Institute built SceneSmith, a system in which three vision-language-model agents design, criticize, and approve simulation-ready rooms for robot training. The team generated more than 1,300 scenes; rooms held up to six times as many objects as earlier baselines, while more than 200 users preferred SceneSmith's realism more than 90 percent of the time. A pretrained policy trained largely on real-world data completed a test in an unseen generated scene, and human evaluators agreed with an automated critic more than 99 percent of the time. Generation still takes multiple hours per scene, and deformable objects remain outside the current system.
Why it matters
SceneSmith coordinates designer, critic, and orchestrator agents to create object-rich 3D rooms that load directly into physics simulators for robot training.
Limits and context
- Generation still takes multiple hours per scene, and deformable objects remain outside the current system.
Key claims
SceneSmith coordinates designer, critic, and orchestrator agents to create object-rich 3D rooms that load directly into physics simulators for robot training.
Qualification: Generation still takes multiple hours per scene, and deformable objects remain outside the current system.
Evidence: source-2026-07-13-020
Sources
- MIT News: SceneSmith agent-generated robot simulation environmentsMIT News · secondary reporting
Corrections
No corrections have been recorded for this story.