Technical Papers Archive

Cross-Layer Energy Analysis of Multimodal Training on Grace Hopper Superchips


Authors: Mahmoud Ahmed and Sameh Abdulah (King Abdullah University of Science and Technology (KAUST)); Olatunji Ruwase (Snowflake, Inc); Sam Ade Jacobs (Microsoft Corporation); Mathis Bode (Forschungszentrum Jülich); and Mohamed Elhoseiny and David E. Keyes (King Abdullah University of Science and Technology (KAUST))

Abstract: Multimodal deep learning jointly processes text, images, and video, but scaling models and sequences shifts training bottlenecks from computation toward memory capacity and data movement. DeepSpeed addresses these limits through CPU offloading, activation checkpointing, and sequence parallelism, yet these techniques add communication and system activity that can alter energy efficiency. We present a cross-layer study of multimodal training on NVIDIA Grace Hopper (GH200) systems, characterizing interactions among the application, runtime, and hardware layers. Using controlled experiments across multiple model scales, we evaluate power, energy, runtime, and throughput under offloading and parallelism strategies. Results show that energy efficiency depends primarily on data movement and compute-communication overlap rather than raw accelerator utilization; consequently, runtime-optimal configurations are not always energy-optimal. We derive practical guidelines for selecting offloading modes, sequence-parallelism degrees, and hardware-aware schedules that balance memory pressure, communication cost, time-to-solution, and energy on tightly coupled heterogeneous systems.



Back to Technical Papers Archive Listing