benchmarks evals
Keep the Browser State When the User Changes the Goal
DynaRev injects additive, unconditional and runtime-conditioned revisions at semantic milestones across 181 web-agent tasks.
Summary
DynaRev injects additive, unconditional and runtime-conditioned revisions at semantic milestones across 181 web-agent tasks.
Web-agent benchmarks usually freeze the user's request before execution, unlike real interactions. DynaRev delivers revisions at task-defined milestones and scores correctness, responsiveness and efficiency across 181 tasks. Five evaluated models often completed only part of the revised requirements; rewriting the instruction while preserving browser state helped, while resetting the browser lost progress. The benchmark supplies a controlled testbed, not evidence that one revision strategy will generalize across deployed sites.
Why it matters
DynaRev injects additive, unconditional and runtime-conditioned revisions at semantic milestones across 181 web-agent tasks.
Limits and context
- Five evaluated models often completed only part of the revised requirements; rewriting the instruction while preserving browser state helped, while resetting the browser lost progress.
- The benchmark supplies a controlled testbed, not evidence that one revision strategy will generalize across deployed sites.
Key claims
DynaRev injects additive, unconditional and runtime-conditioned revisions at semantic milestones across 181 web-agent tasks.
Qualification: Five evaluated models often completed only part of the revised requirements; rewriting the instruction while preserving browser state helped, while resetting the browser lost progress.
Evidence: source-2026-10-31-007
Sources
- arXiv preprint 2610.40133, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.