Model Reviews
Training a 1T-Parameter MoE on EU Public Infrastructure with PyTorch and TorchTitan
A deep dive into how researchers successfully scaled a 1.1 trillion parameter Mixture-of-Experts (MoE) model on shared public supercomputers using the new TorchTitan library and PyTorch's native 4D parallelism.
Read more →