benchmarks evals
The Visual Assistant Enters a Closed Loop
IVG-Bench simulates users and executable tasks across cooking, household manipulation and mobile interfaces.
Summary
IVG-Bench simulates users and executable tasks across cooking, household manipulation and mobile interfaces.
Across 315 scenarios, the best tested model finishes 53.4 percent within the time limit. A human study finds 78.5 percent agreement with the interaction-quality metric, and a small-model training experiment improves task success and quality in the cooking environment.
Why it matters
IVG-Bench simulates users and executable tasks across cooking, household manipulation and mobile interfaces.
Limits and context
No additional limitation was separately recorded.
Key claims
IVG-Bench simulates users and executable tasks across cooking, household manipulation and mobile interfaces.
Evidence: source-2026-11-06-018
Sources
- arXiv preprint 2611.08286, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.