research
Quasi-Newton Training Scaled to 136 Million Parameters
Positive-definite curvature estimates avoided line searches and GPU-unfriendly decompositions.
Summary
Positive-definite curvature estimates avoided line searches and GPU-unfriendly decompositions.
SoftServe builds diagonal and Kronecker-factored curvature estimates that remain positive definite under nonconvexity, using coupled Newton-Schulz matrix operations. It often reached lower losses than Adam, Muon and SOAP on ill-conditioned tasks including a 136-million-parameter physics-informed diffusion model.
Why it matters
Positive-definite curvature estimates avoided line searches and GPU-unfriendly decompositions.
Limits and context
- SoftServe builds diagonal and Kronecker-factored curvature estimates that remain positive definite under nonconvexity, using coupled Newton-Schulz matrix operations.
Key claims
Positive-definite curvature estimates avoided line searches and GPU-unfriendly decompositions.
Qualification: SoftServe builds diagonal and Kronecker-factored curvature estimates that remain positive definite under nonconvexity, using coupled Newton-Schulz matrix operations.
Evidence: source-2026-10-02-020
Sources
- arXiv preprint 2610.02182arXiv · primary research
Corrections
No corrections have been recorded for this story.