Stochastic Gradient Descent Ascent is Suboptimal for Nonconvex-PL Min-Max Games
How far can stochastic gradient descent ascent (SGDA) go by tuning its timescale ratio and step sizes in nonconvex min-max games? We answer this question for nonconvex-PL (NC-PL) games by establishing the first tight complexity of two-timescale SGDA with a fixed timescale ratio and non-increasing…
arXiv - stat.ML2 min read