Know a great AI tool or AI news source? Submit it to AiByts
PyTorch9 min read

Training a 1T-Parameter MoE on EU public infrastructure with PyTorch and TorchTitan

TL;DR Scaling pretraining for a ~1-trillion-parameter Mixture-of-Experts (MoE) model on shared, public infrastructure was previously considered unfeasible due to unique architectural and interconnect bottlenecks. However, by leveraging the fine-grained flexibility...

By The EUROPA consortium (Domyn, Fraunhofer IAIS and IIS)PyTorch Blog
Training a 1T-Parameter MoE on EU public infrastructure with PyTorch and TorchTitan

Read the full story

Originally published by PyTorch Blog on Nov 12, 2026.

Read on PyTorch Blog(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.