Poster Type: Research Posters
Author: Prajwal Singhania (University of Maryland), Emir Gencer (University of Maryland), Siddharth Singh (NVIDIA Corporation), Abhinav Bhatele (University of Maryland)
Abstract: Large language models (LLMs) are increasingly important for scientific computing and agentic coding workflows. With growing model sizes, a single node on current-generation HPC systems no longer suffices for LLM inference. Inference must span slower scale-out interconnects and combine different parallelization strategies. This combinatorial space of strategies is large, and each configuration places different demands on the hardware. Which configuration performs best at a given scale, workload, and model architecture remains unclear. To address this, we empirically sweep hundreds of parallel deployment configurations across models and workloads on two leadership-class systems. We analyze scaling trends and find optimal configurations for different workloads and performance metrics, aiming to provide actionable insights for HPC practitioners.
Best Poster Finalist (BP): yes
Poster: PDF
Poster Summary: PDF