robotics
Describe the Attempt Before Scoring It
D&A separates video description from rubric-based assessment so a vision-language model is less likely to hallucinate robot-task progress.
Summary
D&A separates video description from rubric-based assessment so a vision-language model is less likely to hallucinate robot-task progress.
Vision-language models used as zero-shot reward models often assume the task is succeeding and reduce progress to a binary verdict. D&A first requires a task-grounded description of scene dynamics, then asks for an assessment against an explicit definition of absolute success. That separation anchors the score outside the video's own timeline and makes partial progress and failure easier to distinguish. The authors report better calibration, less confirmation bias and performance matching or exceeding tested post-trained reward models across tasks and embodiments; the scores also supported downstream reinforcement learning. Because the method is prompt-based, its result depends on the underlying model and rubric, and it does not make video-derived rewards automatically safe or correct.
Why it matters
D&A separates video description from rubric-based assessment so a vision-language model is less likely to hallucinate robot-task progress.
Limits and context
- Because the method is prompt-based, its result depends on the underlying model and rubric, and it does not make video-derived rewards automatically safe or correct.
Key claims
D&A separates video description from rubric-based assessment so a vision-language model is less likely to hallucinate robot-task progress.
Qualification: Because the method is prompt-based, its result depends on the underlying model and rubric, and it does not make video-derived rewards automatically safe or correct.
Evidence: source-2026-11-03-012
Sources
- arXiv preprint 2611.03622, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.