benchmarks evals
Coding Agents Compress the Model and Miss Different Things
CompressionBench gives agents weights, a storage cap and inference tests; 29 of 36 submissions pass independent verification.
Summary
CompressionBench gives agents weights, a storage cap and inference tests; 29 of 36 submissions pass independent verification.
The best agent design per task averages a 3.4 percent perplexity increase over BF16 and 77.33 percent downstream accuracy versus 79.56 percent. Full-model feedback reverses 17 of 79 local rankings, and generation tests catch failures perplexity misses.
Why it matters
CompressionBench gives agents weights, a storage cap and inference tests; 29 of 36 submissions pass independent verification.
Limits and context
No additional limitation was separately recorded.
Key claims
CompressionBench gives agents weights, a storage cap and inference tests; 29 of 36 submissions pass independent verification.
Evidence: source-2026-11-05-017
Sources
- arXiv preprint 2611.06858, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.