safety security
The Detector Won the Test. Then It Met the Tool Output.
A prompt-injection audit found that benchmark rankings transferred poorly when detectors screened the longer inputs an agent actually receives.
Summary
A prompt-injection audit found that benchmark rankings transferred poorly when detectors screened the longer inputs an agent actually receives.
The study reconstructed benign tool outputs by replaying ground-truth calls from AgentDojo and tau-bench, then labeled injected versions through differential replay. It evaluated fifteen detectors and two task-aware model judges alongside BIPIA. At a one percent false-positive rate, the best BIPIA detector caught only two percent of AgentDojo injections; another detector caught 72 percent on AgentDojo but 15 percent on tau-bench. Public training data suggested that input form mattered: exposure to isolated attack strings did not establish detection inside a tool response. The authors recommend testing on an agent’s own outputs, reporting detection at low false-positive rates and auditing training overlap. This preprint tests particular detectors and benchmarks; it does not establish that every deployed screening system fails.
Why it matters
A prompt-injection audit found that benchmark rankings transferred poorly when detectors screened the longer inputs an agent actually receives.
Limits and context
- At a one percent false-positive rate, the best BIPIA detector caught only two percent of AgentDojo injections; another detector caught 72 percent on AgentDojo but 15 percent on tau-bench.
- Public training data suggested that input form mattered: exposure to isolated attack strings did not establish detection inside a tool response.
- This preprint tests particular detectors and benchmarks; it does not establish that every deployed screening system fails.
Key claims
A prompt-injection audit found that benchmark rankings transferred poorly when detectors screened the longer inputs an agent actually receives.
Qualification: At a one percent false-positive rate, the best BIPIA detector caught only two percent of AgentDojo injections; another detector caught 72 percent on AgentDojo but 15 percent on tau-bench.
Evidence: source-2026-10-04-002
Sources
- arXiv preprint 2610.03448, original v1arXiv / submitting research authors · primary research
Corrections
: Publication note: This October 4 edition was reconstructed on October 5 after the workstation was offline. It uses original October 2 submissions from the standard Sunday-evening arXiv release, rather than claiming those papers were available for the missed 9 AM run. The date and 9 AM publication slot preserve archive chronology; October 5 developments are excluded. The September 27 Invention Desk is carried over because its replacement was generated October 5.