Research and ACM SRC Posters Archive

From Microbenchmarks to LLM Inference: An End-to-End Analysis on the Energy Efficiency of the Grace Hopper Superchip


Poster Type: Research Posters

Author: Markus Velten (Technical University of Dresden; ZIH, CIDS), Christian Von Elm (Technical University of Dresden; ZIH, CIDS), Lena Jurkschat (Technical University of Dresden; ScaDS.AI, CIDS), Gülçin Gedik (Technical University of Dresden; ZIH, CIDS), Sebastian Döbel (Technical University of Dresden; ScaDS.AI, CIDS), Daniel Hackenberg (Technical University of Dresden; ZIH, CIDS)

Abstract: The NVIDIA Grace Hopper (GH200) Superchip introduces a tightly integrated CPU-GPU architecture with shared power management, enabling new opportunities for energy-efficient high performance computing and AI inference. However, the impact of its power controls across different workloads remains largely unexplored. We present an end-to-end analysis of the GH200, spanning power sensor validation, HPL and HPL-MxP benchmarking, and LLM inference using Llama-3.1-8B. Our study quantifies the interaction between module and GPU power limits, evaluates energy-performance trade-offs, and analyzes operational and total cost of ownership. We demonstrate highly accurate onboard power measurements, identify workload-dependent optimal power settings, and show that mixed precision and inference workloads respond differently to power capping. During the poster session, we will present our experimental methodology, discuss the observed trade-offs, and highlight practical guidelines for energy-efficient deployment of GH200-based HPC and AI systems.

Best Poster Finalist (BP): no
Poster: PDF
Poster Summary: PDF


Back to Poster Archive Listing