research
The Biggest Model Change Wasn’t the Most Causal
Fine-tuning shifted representations in layers that were not necessarily the task drivers.
Published Updated Story ID: mp-2026-09-21-015
Summary
Fine-tuning shifted representations in layers that were not necessarily the task drivers.
A mechanistic study found little correlation between layers with the largest representation changes and components identified as causally important for task performance. Overlap in those components across tasks also did not guarantee transfer.
Why it matters
Fine-tuning shifted representations in layers that were not necessarily the task drivers.
Limits and context
- Overlap in those components across tasks also did not guarantee transfer.
Key claims
Fine-tuning shifted representations in layers that were not necessarily the task drivers.
Qualification: Overlap in those components across tasks also did not guarantee transfer.
Evidence: source-2026-09-21-017
Sources
- arXiv preprint 2609.21113arXiv · primary research
Corrections
No corrections have been recorded for this story.