Focusing on Post-Training
Why I'd invest in post-training existing open-weight LLMs, with Fireworks' Ember-1 as an example of more token-efficient reasoning.
Sebastian Raschka2 min read
Sebastian Raschka, PhD
25 stories
MiMo-V2.6 Pro Architecture and Training Notes
Notes on MiMo-V2.6 Pro's GQA and sliding-window attention, agent training tasks, reward signals, and large RL batches.
Sebastian Raschka1 min readIt's Easy to Dismiss Jev as Just a Classifier
A short note on Jev's generalization, possible encoder-style architecture and training, and Choice and Noul API examples.
Sebastian Raschka2 min readPacing != Pacing Development
My take on AI model pacing as a framework for release checks and the competitive pressure around model releases.
Sebastian Raschka1 min readAI Reasoning Models Course on LinkedIn Learning
A 90-minute LinkedIn Learning course on how reasoning models relate to conventional LLMs and how they are developed.
Sebastian Raschka1 min readOpenAI Astra and Looped Transformers
A short note on OpenAI Astra, recurrent depth, looped transformers, Nanbeige 4.2, and the Mixture-of-Recursions paper.
Sebastian Raschka3 min readReasoning Models From Scratch: Code Setup
A short video on how conventional LLMs relate to reasoning models and agents, along with Python and PyTorch setup using uv.
Sebastian Raschka1 min readTwo Live Book Club Q&As on September 3
Two free live book club Q&A sessions on September 3 about Build a Reasoning Model From Scratch and related reader questions.
Sebastian Raschka1 min readGLM-5.3-Flash Architecture Notes
Architecture note on GLM-5.3-Flash, formerly Ox Alpha, covering its KDA and MLA/DSA hybrid attention, sparse MoE backbone, and four-stream mHC residual path.
Sebastian Raschka1 min readHow Claude's Text Watermarking Works
Short illustration of how Claude's text watermarking is supposed to work based on Anthropic's released materials.
Sebastian Raschka2 min readBuild a Reasoning Model From Scratch Is Now on Amazon
Short note on the Amazon availability of Build a Reasoning Model From Scratch and a warning about counterfeit black-and-white copies sold through Amazon India.
Sebastian Raschka2 min readMuse Glimmer 30B Architecture Notes
Short architecture note on Meta Muse Glimmer 30B, including gated local and global GQA, KV-cache efficiency, and release-time benchmark comparisons.
Sebastian Raschka2 min readLLMs From Scratch Reaches 100,000 GitHub Stars
Short note celebrating the LLMs-from-scratch repository passing 100,000 GitHub stars and summarizing its learning materials.
Sebastian Raschka2 min readKimi K3 Architecture Notes
Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.
Sebastian Raschka2 min readA Few Notable Open-Weight Models This Week
Short note on the architectures of six new open-weight models, including Nanbeige 4.2, Laguna S 2.1, Motif-3-Beta, Solar Open 2, Antares 1B, and BTL-3.
Sebastian Raschka3 min readCorrection for Listing 6.5 in Build a Reasoning Model From Scratch
Short correction note for the random seed in Listing 6.5 on page 198 of Build a Reasoning Model From Scratch.
Sebastian Raschka1 min readInkling: A New Open-Weight 975B MoE with a Few Surprises
Architecture and benchmark notes on Thinking Machines Lab's 975B Inkling MoE, including short convolutions, relative-position bias, training, and effort control.
Sebastian Raschka4 min read200,000 Subscribers
Short note celebrating Ahead of AI reaching 200,000 subscribers.
Sebastian Raschka1 min read
Showing 18 of 25 stories
Load more storiesAiByts Weekly Brief
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts