Workshop: Workshop on Operational Data Analytics in HPC (HPC-ODA)
Authors: Awais Khan (Oak Ridge National Laboratory (ORNL)); Nick Hagerty and Ahmad Maroof Karimi (Oak Ridge National Laboratory); Michael Rowan (AMD); and Jack Lange, Matthew Ezell, Christopher Zimmer, and Scott Atchley (Oak Ridge National Laboratory)
Abstract: Nearly 300 projects use Frontier at the Oak Ridge Leadership Computing Facility each year, with approximately 30% turnover in the project population. These projects execute heterogeneous applications across CPUs and GPUs, and their computational requirements evolve as new workloads enter the facility. Planning next-generation systems requires OLCF lead- ership to better understand accelerator usage across the facility workload population. We present SORT, a Slurm-Orchestrated ROCm Telemetry framework that uses traditional prologue and epilogue hooks to control a ROCprofiler-SDK-based telemetry collector over the lifetime of a Slurm allocation. SORT integrates ROCprofiler-SDK counter collection with the Slurm allocation lifecycle and joins compact counter artifacts with scheduler metadata for offline analysis. Deployed on Borg, SORT collected 88,269 valid counter files associated with 67,373 jobs. The resulting telemetry supports population- level characterization of instruction, memory-request, and FP64 activity and can help identify candidate jobs for deeper profiling.
Back to Workshop on Operational Data Analytics in HPC (HPC-ODA) Archive Listing
Back to Full Workshop Archive Listing