I trained GPT-2 with full-batch gradient descent so you don't have to

The impetus for doing this was provided in large part by Charles Renshaw-Whitman.Learning dynamics people suggested to me that noise was essential in the training of neural networks.