safety security
Compression Hid the Knowledge It Lost
Aggregate accuracy and bias scores concealed subgroup shifts and confident errors across eleven compression methods.
Summary
Aggregate accuracy and bias scores concealed subgroup shifts and confident errors across eleven compression methods.
Researchers evaluated three language models across eleven compression methods and found that compressed systems disproportionately lost head knowledge relative to tail knowledge while often remaining confident about newly incorrect answers. Stable aggregate bias scores also masked opposing movements across demographic subgroups, arguing for granular deployment audits rather than a single perplexity, accuracy or bias number.
Why it matters
Aggregate accuracy and bias scores concealed subgroup shifts and confident errors across eleven compression methods.
Limits and context
No additional limitation was separately recorded.
Key claims
Aggregate accuracy and bias scores concealed subgroup shifts and confident errors across eleven compression methods.
Evidence: source-2026-08-23-006
Sources
- arXiv preprint 2608.19670arXiv · primary research
Corrections
No corrections have been recorded for this story.