Workshops Archive

HOLOS: A Gradient-Descent-Based Design-Space Exploration Framework for Hierarchical AI Accelerator Systems


Workshop: AI on HPC: Performance Engineering, Challenges and Opportunities

Authors: Sairam Sri Vatsavai (Brookhaven National Laboratory); Daniel Peralta Velazquez, Siyuan Ma, George Biros, Jaydeep Kulkarni, Lizy Kurian John, and Andreas Gerstlauer (The University of Texas at Austin); and Adolfy Hoisie and Lingda Li (Brookhaven National Laboratory)

Abstract: The rapid growth of large language models has increased demand for hierarchical accelerator systems spanning processing elements, tiles, accelerators, and clusters. Exploring these systems is challenging because hardware parameters and software mappings interact across multiple levels. We present a gradient based design space exploration framework that unifies hardware and software co design using compact tiling factors for data, pipeline, and tensor parallelism together with per accelerator PE array and tile dimensions. The minimum required hardware is inferred directly from each mapping, reducing a conventional O(hardwarexmappings) search to O(mappings). A differentiable per operation model estimates latency, area, and energy. Compared with Calculon, our model achieves 0.99 Pearson correlation and approximately 4 percent mean absolute percentage error across 13B to 540B parameter LLMs. Under realistic memory and interconnect constraints for GPU system, the search reaches within approximately 1 percent of optimal energy delay product while evaluating 27 to 64 times fewer designs.


Back to AI on HPC: Performance Engineering, Challenges and Opportunities Archive Listing

Back to Full Workshop Archive Listing