machine learning1 min read
I merged attention and MLP processing into a single gated Transformer block. GAP [R]
I've been experimenting with an alternative Transformer architecture called GAP (Gated Attention Projection).
By /u/WAMFTReddit - r/ML
Read the full story
Originally published by Reddit - r/ML on Nov 8, 2026.
Comments
Sign in to join the discussion.