safety security
The Harness Is Part of the Security Boundary
Paired usability tests separate genuine web-agent defenses from agents that merely fail to complete the task.
Summary
Paired usability tests separate genuine web-agent defenses from agents that merely fail to complete the task.
WebTrap Farm evaluates malicious user prompts, prompt injection and deceptive page design across the four stages of a web-agent harness: goal interpretation, perception, planning and execution. Paired usability tasks distinguish a safe refusal from inability to act. The authors report unsafe behavior even without explicit attack content and find statistically significant effects from both the harness and the backend model. Their analysis localizes where defenses emerge in the workflow. The abstract does not provide universal rates or name a production incident, but it shows why model-only comparisons miss deployed-system risk.
Why it matters
Paired usability tests separate genuine web-agent defenses from agents that merely fail to complete the task.
Limits and context
- The abstract does not provide universal rates or name a production incident, but it shows why model-only comparisons miss deployed-system risk.
Key claims
Paired usability tests separate genuine web-agent defenses from agents that merely fail to complete the task.
Qualification: The abstract does not provide universal rates or name a production incident, but it shows why model-only comparisons miss deployed-system risk.
Evidence: source-2026-11-10-014
Sources
- arXiv preprint 2611.11971, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.