Workshops Archive

Invited Talk: Torsten Hoefler: AI Infrastructure at Hyperscale - The Role of (Ultra) Ethernet


Workshop: ExaMPI26: Workshop on Extreme-Scale MPI

Authors: Nikoli Dryden (Lawrence Livermore National Laboratory (LLNL)), Torsten Hoefler (ETH Zürich)

Abstract: The rapid rise of generative AI is driving an unprecedented expansion of hyperscale infrastructure. Modern AI services generate trillions of tokens per day, requiring ever larger clusters and making efficiency a first-order challenge. While specialized accelerators such as Maia 200 are essential for delivering performant and cost-effective inference, their capabilities can only be fully realized through a scalable and efficient network fabric. This talk explores the co-design of next-generation AI infrastructure, from accelerator architecture and large-scale inference systems to the communication challenges posed by massive distributed workloads. I will argue that networking has become the defining system bottleneck and discuss the role of Ethernet in addressing it. Finally, I will present the vision behind Ultra Ethernet and its evolution into a high-performance, open networking platform capable of supporting the communication demands of future AI and HPC systems at hyperscale.


Back to ExaMPI26: Workshop on Extreme-Scale MPI Archive Listing

Back to Full Workshop Archive Listing