Yannic Kilcher Blog

I BUILT A FULLY AUTOMATIC MANSPLAINER

All information about GTC and the DGX Spark Raffle is here: https://www.ykilcher.com/gtcLinks:Homepage: https://ykilcher.comMerch: https://ykilcher.com/merch...

Traditional X-Mas Stream

Traditional Holiday Live Stream

https://ykilcher.com/discordLinks:TabNine Code Completion (Referral): http://bit.ly/tabnine-yannickYouTube: https://www.youtube.com/c/yannickilcherTwitter: h...

TiDAR: Think in Diffusion, Talk in Autoregression (Paper Analysis)

Paper: https://arxiv.org/abs/2511.08923Abstract:Diffusion language models hold the promise of fast parallel generation, while autoregressive (AR) models typi...

Titans: Learning to Memorize at Test Time (Paper Analysis)

Paper: https://arxiv.org/abs/2501.00663Abstract:Over more than a decade there has been an extensive research effort on how to effectively utilize recurrent m...

[Paper Analysis] The Free Transformer (and some Variational Autoencoder stuff)

https://arxiv.org/abs/2510.17558Abstract:We propose an extension of the decoder Transformer that conditions its generative process on random latent variables...

[Video Response] What Cloudflare's code mode misses about MCP and tool calling

Theo's Video: https://www.youtube.com/watch?v=bAYZjVAodooCloudflare article: https://blog.cloudflare.com/code-mode/Links:Homepage: https://ykilcher.comMerch:...

[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)

Paper: https://arxiv.org/abs/2508.21038Abstract:Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nasc...

AGI is not coming!

jack Morris's investigation into GPT-OSS training datahttps://x.com/jxmnop/status/1953899426075816164?t=3YRhVQDwQLk2gouTSACoqA&s=09

Context Rot: How Increasing Input Tokens Impacts LLM Performance (Paper Analysis)

Paper: https://research.trychroma.com/context-rotAbstract:Large Language Models (LLMs) are typically presumed to process context uniformly—that is, the model...

Energy-Based Transformers are Scalable Learners and Thinkers (Paper Review)

Paper: https://arxiv.org/abs/2507.02092Code: https://github.com/alexiglad/EBTWebsite: https://energy-based-transformers.github.io/Abstract:Inference-time com...

On the Biology of a Large Language Model (Part 2)

An in-depth look at Anthropic's Transformer Circuit Blog PostPart 1 here: https://youtu.be/mU3g2YPKlsADiscord here: https;//ykilcher.com/discordhttps://trans...

On the Biology of a Large Language Model (Part 1)

An in-depth look at Anthropic's Transformer Circuit Blog Posthttps://transformer-circuits.pub/2025/attribution-graphs/biology.htmlAbstract:We investigate the...

[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

#deepseek #llm #grpoGRPO is one of the core advancements used in Deepseek-R1, but was introduced already last year in this paper that uses a combination of n...

Traditional Holiday Live Stream

https://ykilcher.com/discordLinks:TabNine Code Completion (Referral): http://bit.ly/tabnine-yannickYouTube: https://www.youtube.com/c/yannickilcherTwitter: h...