Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules

Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference requires increasingly large KV caches.