Workshop: HUST-26: 13th International Workshop on HPC User Support Tools
Authors: Jingyan Jiang (Argonne National Laboratory); Akhilesh Bondapalli (Clemson University, Argonne National Laboratory); Krishna Rauniyar (University of Louisiana at Lafayette, Argonne National Laboratory); Murat Keceli, Yasaman Ghadar, Haritha Siddabathuni Som, Venkatram Vishwanath, and Michael Papka (Argonne National Laboratory); Rong Ge (Clemson University); and Huihuo Zheng (Argonne National Laboratory)
Abstract: AI question-answering assistants increasingly serve user support at high-performance computing (HPC) facilities, but they offer no systematic way to judge answer quality in production or attribute a bad answer to the component at fault, and they stay static after deployment apart from knowledge-base updates.
We present AskALCF, an agentic log-analysis and improvement pipeline that scores each production interaction, identifies whether a failure lies in missing knowledge, retrieval, prompting, model behavior, or tool execution, and routes it to an actionable repair. Its diagnostic core pairs answer-quality assessment with a two-scope sufficiency test separating information absent from the corpus from information present but never retrieved. Controlled harness training, supervised by expert references and null-edit controls, improves the pipeline itself.
Across 2,174 support turns AskALCF reduced unattributable failures from 41.4% to 14.2%, and harness training raised agreement with expert labels from 108/150 to 123/150 turns, a governed self-evolution driven by production evidence.
Back to HUST-26: 13th International Workshop on HPC User Support Tools Archive Listing
Back to Full Workshop Archive Listing