infrastructure
The Last Layer Was Enough to Choose the Training Data
LESSER approximates full-gradient selection with output-layer gradients available from a forward pass.
Summary
LESSER approximates full-gradient selection with output-layer gradients available from a forward pass.
LESSER wraps gradient-based post-training data selectors but uses output-layer gradients instead of a full backward pass for every candidate. The authors report 9.7-times lower feature-extraction FLOPs for supervised fine-tuning and three-times lower cost on reinforcement-learning benchmarks while tracking full-gradient downstream performance. Individual rankings differed, but selected batches remained aligned.
Why it matters
LESSER approximates full-gradient selection with output-layer gradients available from a forward pass.
Limits and context
No additional limitation was separately recorded.
Key claims
LESSER approximates full-gradient selection with output-layer gradients available from a forward pass.
Evidence: source-2026-10-05-006
Sources
- arXiv preprint 2610.03702arXiv · primary research
Corrections
No corrections have been recorded for this story.