safety security
The Causal Mask Passed While Future Tokens Leaked Elsewhere
A full-sequence routing summary made validation loss look dramatically better even as free-running generation deteriorated.
Summary
A full-sequence routing summary made validation loss look dramatically better even as free-running generation deteriorated.
Prefix-invariance checks localized the leak to a dynamic routing-query path outside causal attention. Two repaired models lost the apparent advantage and returned to the control range, showing why teacher-forced loss needs whole-model causal and generation audits.
Why it matters
A full-sequence routing summary made validation loss look dramatically better even as free-running generation deteriorated.
Limits and context
No additional limitation was separately recorded.
Key claims
A full-sequence routing summary made validation loss look dramatically better even as free-running generation deteriorated.
Evidence: source-2026-10-23-021
Sources
- arXiv preprint 2610.31522, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.