MoE2 min read
Multi-GPU MoE Inference with Adaptive Expert Offloading
Modern Mixture-of-Experts (MoE) inference is constrained by GPU memory. This constraint is mitigated by expert offloading, which stores part of the model's weight in host memory and executes on demand.
By William X. Zheng, Zhongjie Chen, Shuo Yang, Ziming Mao, Xiaoze Fan, Yi Pan, Kunjun Li, Hanfei Yu, Chenfeng Xu, Yang Zhou, Rishabh Iyer, Ion StoicaarXiv - cs.AI
Read the full story
Originally published by arXiv - cs.AI on Nov 10, 2026.
Comments
Sign in to join the discussion.