Modular Tech Talk: Frontier-scale MoE Serving at Modular

Modular Tech Talks is an exclusive series featuring internal presentations from our product and engineering teams explaining the inner workings of the Modular technology stack. In this edition, Brendan Duke presents on frontier-scale Mixture of Experts (MoE) serving at Modular. MoE enhances the trade-off between quality and performance in Large Language Models (LLMs).