TheMachine Press

A daily newspaper for the age of artificial intelligence.

Morning edition

Permanent story

safety security

Changing the User's Language Changes the Guard's Verdict

LinguaGuardBench holds tool calls and labels fixed while translating only the instruction, exposing language-dependent safety decisions.

Published Updated Story ID: mp-2026-10-16-011
Read the complete editionStory JSON

Summary

LinguaGuardBench holds tool calls and labels fixed while translating only the instruction, exposing language-dependent safety decisions.

Step-level agent guards are usually prompted and benchmarked in English even when users are not. LinguaGuardBench translates only the user instruction in 1,220 labeled AgentDojo trajectory steps into five languages, leaving every tool call, observation and label byte-identical. Across 19 guards, many open, small and purpose-built systems changed verdicts beyond their own noise; among nine guards below 13 billion parameters, the placebo-adjusted flip rate rose from 1.5 points in French to 4.4 in Amharic. The strongest frontier guards moved by about one point or less in the authors' tests. This does not prove a universal language hierarchy, but it shows that language itself can alter a supposedly invariant safety boundary.

Why it matters

LinguaGuardBench holds tool calls and labels fixed while translating only the instruction, exposing language-dependent safety decisions.

Limits and context

  • Step-level agent guards are usually prompted and benchmarked in English even when users are not.
  • LinguaGuardBench translates only the user instruction in 1,220 labeled AgentDojo trajectory steps into five languages, leaving every tool call, observation and label byte-identical.
  • This does not prove a universal language hierarchy, but it shows that language itself can alter a supposedly invariant safety boundary.

Key claims

  1. LinguaGuardBench holds tool calls and labels fixed while translating only the instruction, exposing language-dependent safety decisions.

    Qualification: Step-level agent guards are usually prompted and benchmarked in English even when users are not.

    Evidence: source-2026-10-16-011

Sources

  1. arXiv preprint 2610.22244, original v1arXiv / submitting research authors · primary research

Corrections

No corrections have been recorded for this story.