research
The Nearby Expert May Be a Voting Effect
Random output-logit bias matched or beat full-weight perturbations on most tested reasoning tasks while keeping the model frozen.
Summary
Random output-logit bias matched or beat full-weight perturbations on most tested reasoning tasks while keeping the model frozen.
The Neural Thickets hypothesis attributes gains from random weight perturbations to dense neighborhoods of task-specialized models. A reproduction finds a simpler explanation: perturbing one decoder block recovers much of the gain, while adding random bias only to output logits matches or exceeds full-weight perturbations on most tested reasoning tasks. The authors formalize Perturb-Select-Vote, selecting strong perturbed predictors on a small validation set and aggregating them. Vision tests under domain shift also found little extra benefit from changing all the weights. The result challenges one geometric interpretation; it does not show that every perturbation ensemble reduces to logit voting.
Why it matters
Random output-logit bias matched or beat full-weight perturbations on most tested reasoning tasks while keeping the model frozen.
Limits and context
- A reproduction finds a simpler explanation: perturbing one decoder block recovers much of the gain, while adding random bias only to output logits matches or exceeds full-weight perturbations on most tested reasoning tasks.
- The result challenges one geometric interpretation; it does not show that every perturbation ensemble reduces to logit voting.
Key claims
Random output-logit bias matched or beat full-weight perturbations on most tested reasoning tasks while keeping the model frozen.
Qualification: A reproduction finds a simpler explanation: perturbing one decoder block recovers much of the gain, while adding random bias only to output logits matches or exceeds full-weight perturbations on most tested reasoning tasks.
Evidence: source-2026-11-01-003
Sources
- arXiv preprint 2610.40649, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.