Doctoral Showcase Archive

Embrace the Chaos: Harnessing Variability in Accelerator-Rich Systems


Author: Rutwik Jain (University of Wisconsin-Madison)

Advisor: Matthew Sinclair (University of Wisconsin-Madison), Shivaram Venkataraman (University of Wisconsin-Madison)

Abstract: HPC and ML workload scaling have led supercomputers to deploy large, heterogeneous systems with extreme power limits. Current GPU thermal design power limits (TDPs) range from 1,400–2,000W, scaling to extreme power budgets at scale. Thus, power has displaced performance as the dominant constraint. My research addresses this challenge by co-optimizing power and performance at scale through cross-stack approaches spanning hardware, firmware, systems, and applications.

Power management (PM) causes performance variation across accelerators, making it hard for users to get repeatable performance and for administrators to balance resource utilization. I was the first to characterize GPU variability across production clusters, uncovering consistent variability. This work guided MLCommons' benchmarking effort on handling variability. To harness this variability, I developed PAL, a cluster scheduler that leverages application-specific variability during job placement to optimize latencies and resource utilization.

Further investigating PM, we found GPU power often spikes above TDP (temporarily), triggering frequency throttling that hurts performance. I investigated these power excursions, especially frequent in compute-intensive workloads, and developed solutions that reduce excursions and improve latency.

As workloads rapidly evolve, new workloads often show different excursion characteristics as well as power/performance variability. To adapt our solutions to new, unseen workloads, I developed Minos, which groups similar workloads using power and utilization characteristics while reducing profiling overheads.

Overall, my research demonstrates the critical impact of excursions and GPU variability on large-scale systems. As power constraints intensify and hardware limits are pushed, this cross-stack co-design offers essential techniques to improve power-constrained supercomputing in the future.


Thesis Canvas: pdf



Back to Doctoral Showcase Archive Listing