SGD2 min read
Where, When, and How Many Bits? Scaling Laws for SGD with Stochastic Rounding
Low-precision training reduces computation but introduces additional rounding error. We theoretically study how stochastic rounding (SR) affects prediction risk as model size, training time, and precision change.
By Zhichao Wang, Michael W. MahoneyarXiv - stat.ML
Read the full story
Originally published by arXiv - stat.ML on Nov 2, 2026.
Comments
Sign in to join the discussion.