Sometimes less is more: Investigating inductive biases of MLPs and Transformers on noisy k-parity features
We empirically investigate how MLPs and Transformers differ when trained on data sets containing one or more features encoded by noisy k-parity. Noisy k-parity gives us hard synthetic data sets in which we can control the predictivity and availability of every feature.
By Alasdair ParenarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Oct 30, 2026.
Comments
Sign in to join the discussion.