When can Optimally Trained Transformers Achieve Near-Lossless Token Prediction for non-Markovian Data?
There exists little mathematical theory explaining why Transformer architectures are empirically successful even for non-Markovian data. To this end, this paper studies next-token prediction via stochastic optimal control theoretically trained Transformers for such sources, where we consider either…
By Ka\u{g}an Akman, Jake Feldman Starosta, Naci Saldi, Serdar Y\"ukselarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Nov 2, 2026.
Comments
Sign in to join the discussion.