benchmarks evals
The Video Had to Obey the Program
PROWBench checks generated scenes against timestamped world records, including events outside the camera view.
Summary
PROWBench checks generated scenes against timestamped world records, including events outside the camera view.
PROWBench contains 170 programmatically constructed episodes and 600 proxy videos with logged entity states and timestamped events. Because the record persists beyond any one camera view, evaluators can test whether a generated video realizes the observable consequences of the actual program rather than merely looking plausible. The benchmark adds metrics for entity control, long-horizon memory, logic-render alignment and interaction success; it evaluates adherence, not whether a model is a complete game engine.
Why it matters
PROWBench checks generated scenes against timestamped world records, including events outside the camera view.
Limits and context
- The benchmark adds metrics for entity control, long-horizon memory, logic-render alignment and interaction success; it evaluates adherence, not whether a model is a complete game engine.
Key claims
PROWBench checks generated scenes against timestamped world records, including events outside the camera view.
Qualification: The benchmark adds metrics for entity control, long-horizon memory, logic-render alignment and interaction success; it evaluates adherence, not whether a model is a complete game engine.
Evidence: source-2026-10-03-013
Sources
- arXiv preprint 2610.02205arXiv · primary research
Corrections
No corrections have been recorded for this story.