PyTorch9 min read
Training a 1T-Parameter MoE on EU public infrastructure with PyTorch and TorchTitan
TL;DR Scaling pretraining for a ~1-trillion-parameter Mixture-of-Experts (MoE) model on shared, public infrastructure was previously considered unfeasible due to unique architectural and interconnect bottlenecks. However, by leveraging the fine-grained flexibility...
By The EUROPA consortium (Domyn, Fraunhofer IAIS and IIS)PyTorch Blog

Read the full story
Originally published by PyTorch Blog on Nov 12, 2026.
Comments
Sign in to join the discussion.