infrastructure
Expert Pruning Rebuilds What It Removes
A continuous selection relaxation and lightweight reconstruction preserved more performance under aggressive cuts.
Summary
A continuous selection relaxation and lightweight reconstruction preserved more performance under aggressive cuts.
OptEP selects a fixed expert subset by optimizing reconstruction of the original mixture-of-experts layer, then updates retained weights. Across several models and reasoning tasks, reported absolute gains over pruning baselines reached 11.06 to 16.20 percentage points, depending on architecture. These are benchmark results, not universal deployment savings.
Why it matters
A continuous selection relaxation and lightweight reconstruction preserved more performance under aggressive cuts.
Limits and context
- These are benchmark results, not universal deployment savings.
Key claims
A continuous selection relaxation and lightweight reconstruction preserved more performance under aggressive cuts.
Qualification: These are benchmark results, not universal deployment savings.
Evidence: source-2026-10-21-020
Sources
- arXiv preprint 2610.27123, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.