Technical Papers Archive

EMO: Energy Efficiency Modeling and Optimization for AI Workloads


Authors: Jiyu Luo, Shaoyu Chen, and Jingwei Sun (University of Science and Technology of China); Shengcai Liu and Ke Tang (Southern University of Science and Technology, Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation); and Guangzhong Sun (University of Science and Technology of China)

Abstract: The massive energy consumption of GPU-accelerated AI workloads challenges sustainable computing. We observe that execution asynchrony (e.g., CPU-GPU, concurrent streams, and multi-GPU) creates slack, allowing non-critical kernels to run at lower frequencies to save energy with minimal performance impact. However, existing approaches struggle to achieve both workload generality and fine-grained slack discovery, while high-fidelity modeling incurs substantial overhead.

We present EMO, a lightweight framework for exploiting these opportunities. EMO constructs a low-level dependency graph capturing execution asynchrony and performs what-if timing analysis to identify slack windows. It then introduces dependency-aware kernel packing, which preserves critical paths while collapsing redundant details, enabling high-fidelity latency-energy modeling with minimal profiling cost. Finally, EMO combines graph analysis and pack-level models to formulate and solve a constrained combinatorial optimization problem for frequency policies under given latency targets. Evaluations show that EMO reduces energy consumption by 15%–28% with only 2%–5% performance loss and negligible overhead.



Presentation: file

Back to Technical Papers Archive Listing