Birds of a Feather Archive

Operational Data Analytics: Agentic AI for HPC Facilities


Authors: Dong Dai (University of Delaware), Kevin Menear (National Laboratory of the Rockies (NLR)), Avisek Naug (HPE), Keiji Yamamoto (RIKEN Center for Computational Science (R-CCS)), Steven Farrell (Lawrence Berkeley National Laboratory (LBNL)), Jeff Hanson (HPE), Kathleen Shoga (Lawrence Livermore National Laboratory (LLNL)), Melissa Romanus (Lawrence Berkeley National Laboratory (LBNL)), Michael Ott (Leibniz Supercomputing Centre (LRZ), Energy Efficient High Performance Computing Working Group (EE HPC WG)), Nil Mu (Arizona State University), Paul Brunk (University of Georgia), Tim Osborne (Oak Ridge National Laboratory (ORNL)), Woong Shin (Oak Ridge National Laboratory (ORNL))

Abstract: Operational data analytics (ODA) has become essential for understanding and optimizing HPC systems. However, converting the vast amounts of operational data they generate into actionable insights still requires significant manual effort and domain expertise. Agentic AI has the potential to extend traditional analytics by reading telemetry, correlating signals, diagnosing problems, and drafting fixes for an operator to approve. This BoF will explore the state of the art in agentic AI’s applicability to ODA, promising use cases, limitations, and governance challenges. Participants will share experiences and discuss how agentic AI can enhance future HPC operations and efficiency.

Long Description: * Background This BoF builds on previously held BoFs at SC and ISC and is being organized by members of the ODA team within the Energy Efficient HPC Working Group (EEHPCWG). The ODA team comprises practitioners in HPC operations who deal with monitoring and data analytics on a daily basis. This BoF covers several challenging topics in the frontier of ODA every year, starting from ODA tools, frameworks and building blocks, data analysis methods, approaches, AI/ML application, data standardization, and closing the gap between ODA research and operations. This BoF establishes a larger community that is backed by the EEHPCWG, fosters collaboration among different HPC sites, and drives further discussion towards best practices while advancing the field.

* Goal This BoF brings facility operators, systems researchers, and AI practitioners together to discuss where LLM-based agents can help run HPC facilities, what evidence makes their recommendations trustworthy, and how to evaluate them. We also want to identify and understand what gaps and challenges exist in letting agents be more involved and active in operational decisions.

* Topic Every large HPC site already records vast amounts of operational data: job logs, I/O and network traces, power and cooling telemetry, scheduler state, and maintenance records. Sites have been developing and deploying various data analytics approaches (ML/DL/Optimization) for pre-run job characterization and misconfiguration detection, log analysis and job classification, anomaly triage, and I/O diagnosis. These tools surface signals, but an expert still has to read them, connect them, and act.

LLM-based AI agents are the emerging capability that can address this data to action gap, where an agent can query telemetry, correlate signals across sources, reasons about its meaning, propose a diagnosis with its evidence, and draft a fix for an operator to approve. Several sites have begun exploring this, starting with chatbot interfaces over existing analysis tools and moving towards agents that invoke those tools directly. However, the work is early and fragmented. It would be better to compare notes before each site hardens its own silo.

* Outcome This BoF continues to serve an established community of facility operators, researchers, and data analytics experts, and extends its scope to AI- and agent-supported analysis and action. Sites that have started building agents will share what worked and what did not. Sites that have not started will leave knowing what a realistic first step looks like and how to engage with this community. As a tangible outcome, we will capture the content of the BoF in a technical report and publish it via the Energy Efficient HPC Working Group.


Back to Birds of a Feather Archive Listing