A Quadratic Lens on Muon's Polar Update: Benefits, Pitfalls, and the Role of Newton-Schulz
Muon updates matrix-valued parameters along approximately orthogonalized directions and often trains large language models faster than AdamW, yet it remains unclear when and why orthogonalization helps.
By Egor Shulgin, Sam Laing, Antonio Orvieto, Peter Richt\'arikarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Oct 23, 2026.
Comments
Sign in to join the discussion.