Mixture of Experts (MoE)

5 stories

Stories

Efficient MoE Training for Biological Foundation Models
As language models grow, scaling dense architectures becomes increasingly expensive. In a dense transformer, every token passes through every layer, so adding...
NVIDIA Developer BlogSep 24, 20265 min read
Translating CUDA Tile Operations from Python to Rust Using Agentic AI
cuTile Rust (cutile-rs) is a tile-based system for safe, idiomatic GPU kernel authoring in the Rust programming language. Extending the Rust ownership model to...
NVIDIA Developer BlogSep 16, 202617 min read
Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each
How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the...
NVIDIA Developer BlogSep 15, 20267 min read
Accelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine
Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE...
NVIDIA Developer BlogSep 14, 202611 min read
When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving
Encode-prefill-decode (EPD) disaggregation is an inference optimization technique for multimodal models that separates the vision encoder stage from the prefill...
NVIDIA Developer BlogSep 9, 202610 min read