Workshops Archive

Compositional Energy Modeling of Layer-Resolved GPU Training


Workshop: Sustainable Supercomputing

Authors: Amritanshu Verma and Sarah Neuwirth (Johannes Gutenberg University. Mainz)

Abstract: We present a compositional energy model for layer-resolved GPU training on AMD MI300X and NVIDIA H100 datacenter GPUs. Instead of fitting whole-run energy, the model assigns energy coefficients to hardware work classes, including matrix-engine convolutions, GEMM kernels, bandwidth-bound layers, gradient all-reduce, and static device power. Coefficients are calibrated in situ from steady-state replays of production ResNet-50 training kernels, then composed over full runs whose classes interleave, overlap, and contend for shared resources. On MI300X, coefficients calibrated on a native HIP implementation transfer to unmodified PyTorch without refitting, predicting energy to -7.3% at one GPU and +8.5% at sixteen GPUs. On H100, a frequency-dependent active-state term scoped by matrix-pipe utilization gives 9.0% mean absolute error over 42 operating points and 6.4% where the architecture took part in no modeling decision. Residuals identify missing vector work, state power, and operating-point effects. Power-cap sweeps show that per-operation energy falls by 20-41% from 750 W to 450 W on MI300X, so coefficients must be remeasured or corrected when the operating point changes.


Back to Sustainable Supercomputing Archive Listing

Back to Full Workshop Archive Listing