Mixture of Wide Experts: Sparse Routing for Interpretable Native Coordinates
Language models often spread a concept across many internal dimensions, making their features difficult to interpret. Sparse autoencoders (SAEs) recover clearer features after training. We ask whether models can instead learn interpretable output coordinates directly.
By Tavish Mankash, Vardhaman Kalloli, Keshava Prasad, Deepan MuthirayanarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Nov 5, 2026.
Comments
Sign in to join the discussion.