Workshops Archive

A case study on the resource utilisation and queue dynamics of an institutional HPC system


Workshop: Workshop on Operational Data Analytics in HPC (HPC-ODA)

Authors: Syed Ibtisam Tauhidi, Steven Cousens, David Laverty, David Smyth, Bronis de Supinski, and Hans Vandierendonck (Queen's University Belfast)

Abstract: HPC systems are often evaluated using aggregate metrics that implicitly assume a uniform user population. Using a year-long trace from an institutional HPC system, spanning 4.2 million jobs and 596 users, we show this assumption fails for demand-concentrated workloads: a small cohort drives most computational demand, energy consumption, and the mismatch between requested and actually-used resources. This concentration produces severe queue unfairness governed by job shape: small, backfill-compatible jobs dispatch instantly regardless of priority, while larger jobs do not, so the users best served by the scheduler are not those with the least demand, but those whose jobs fit available capacity. Counterfactual simulation shows targeted interventions recover most of the achievable benefit: resizing resource requests cuts mean queue wait by 67%, and pacing the heaviest 5% of submitters cuts mean wait by 82%. Both the waste and the queue risk are predictable from submission-time metadata alone, enabling proactive management.


Back to Workshop on Operational Data Analytics in HPC (HPC-ODA) Archive Listing

Back to Full Workshop Archive Listing