Workshops Archive

Custodian: Enabling Automatic Performance Engineering Agents on the NERSC HPC System


Workshop: PMBS26: The 17th International Workshop on Performance Modeling, Benchmarking, and Simulation of High Performance Computer Systems

Authors: AbdulHannan Mohammed, Oscar Antepara, Dhruva Kulkarni, Muaaz Gul Awan, Samuel Williams, and Jack Deslippe (Lawrence Berkeley National Laboratory)

Abstract: The advancement of Large Language Models and the rapid evolution of AI assistants and agents create an opportunity for the development of highly specialized HPC agents to enable automatic performance analysis. In this paper, we present Custodian, an HPC performance analysis agent that analyzes previously collected job telemetry and application-output data to characterize application behavior, identify potential performance bottlenecks, and transform performance metrics into natural language descriptions and figures that suggest optimizations or highlight computing inefficiencies. We validate Custodian’s telemetry-derived application states and roofline analyses against NVIDIA Nsight Systems and Nsight Compute across four representative applications executed on NERSC Perlmutter. Finally, we discuss the tradeoffs, cost, productivity and systems applicability of HPC agents on the NERSC HPC system.


Back to PMBS26: The 17th International Workshop on Performance Modeling, Benchmarking, and Simulation of High Performance Computer Systems Archive Listing

Back to Full Workshop Archive Listing