MLP3 min read
Transformer MLP Gate Thresholds Are Couplings to a Carried Reference Direction
The corpus-mean direction of a transformer's residual stream is a component shared across all inputs, and is commonly removed by mean-centering before representational analysis.
By Olli TuomiarXiv - cs.CL (NLP)
Read the full story
Originally published by arXiv - cs.CL (NLP) on Sep 30, 2026.
Comments
Sign in to join the discussion.