benchmarks evals
The Verifier Goes Looking for the Missing Evidence
VeriCUA treats completion checking as an exploration task, choosing GUI or code checks for unresolved requirements instead of judging a fixed screenshot.
Summary
VeriCUA treats completion checking as an exploration task, choosing GUI or code checks for unresolved requirements instead of judging a fixed screenshot.
Computer-use agents are often graded from partial observations: a vision-language model sees only supplied screenshots, while generated checking code sees only accessible application state. VeriCUA first builds a task rubric, then actively selects GUI or code inspections for requirements that remain unresolved and combines the findings under explicit evidence rules. Across desktop, mobile, web, online and offline settings, the authors report better agreement with reference labels than the compared baselines, with the largest offline gains on hard cases and stronger rejection of false online completion claims. The preprint is explicitly work in progress, and the abstract does not establish that exploration eliminates verifier blind spots.
Why it matters
VeriCUA treats completion checking as an exploration task, choosing GUI or code checks for unresolved requirements instead of judging a fixed screenshot.
Limits and context
- Computer-use agents are often graded from partial observations: a vision-language model sees only supplied screenshots, while generated checking code sees only accessible application state.
- VeriCUA first builds a task rubric, then actively selects GUI or code inspections for requirements that remain unresolved and combines the findings under explicit evidence rules.
- The preprint is explicitly work in progress, and the abstract does not establish that exploration eliminates verifier blind spots.
Key claims
VeriCUA treats completion checking as an exploration task, choosing GUI or code checks for unresolved requirements instead of judging a fixed screenshot.
Qualification: Computer-use agents are often graded from partial observations: a vision-language model sees only supplied screenshots, while generated checking code sees only accessible application state.
Evidence: source-2026-10-30-003
Sources
- arXiv preprint 2610.40578, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.