Scaling and Renormalization in Neural Networks: An Effective Renormalization-Group Description of Grokking
Grokking is a delayed transition from memorization to generalization in deep neural networks (NNs). We examine whether scaling and renormalization group (RG) theories can describe this dynamics, establishing a quantitative physical framework for neural generalization transitions.
arXiv cs.LG2 min read