Birds of a Feather Archive

Inference Services and Agents on HPC: Scaling Scientific AI for Academic Community


Authors: Dan Stanzione (The University of Texas at Austin), Weicheng Huang (National Center for High-Performance Computing (NCHC), Taiwan), Venkatram Vishwanath (Argonne National Laboratory), Jason Haga (National Institute of Advanced Industrial Science & Technology), Frank Würthwein (San Diego Supercomputing Center (SDSC)), Thomas Hauser (NSF National Center for Atmospheric Research (NCAR)), Séverine Habert (NVIDIA Corporation), Ajay Navilarekal (Leibniz Supercomputing Centre (LRZ))

Abstract: Initiated at SC25 to explore AI inference services in the academic HPC community, this BoF returns to address the fast-evolving landscape of inference services, agentic AI, tool services, and AI4Science. This practitioner forum brings centers and researchers together to share practical insights and experiences spanning infrastructure, software, and use cases. This session aims to cultivate collaborative solutions for AI developments. The goal remains to foster a sustained, cross-institutional "community of practice" bridging academic and governmental HPC centers and the user community to foster collaborations in this area.

Long Description: AI inferencing capability is increasingly important inside academia for both inference and training. Since 2025, as this BoF was held in SC25, inferencing services have continued to advance rapidly to include Agentic AI and co-AI- scientists. Agents can be seen as inference in a loop, repeatedly calling served models to plan, generate code, and act on the system, and their bursty, partly non-deterministic tool calls must be sandboxed so autonomous actions cannot damage shared systems. This is a very different proposition from chatbot, and brings new requirements on the infrastructure in terms of security and performance. In the past year, our cohort also explored these critical topics across various venues and worked closely with various international activities, such as HPC Asia/SCA2026, TPC26 and ADAC18 etc. to further strengthen the trust between members. By sharing inference infrastructure, use cases, best practices, agent skills, agent orchestration, software environment/stacks via a dynamic format of lightning talks, live polls, and interactive panel discussions, this BoF would like to cultivate collaborative solutions with modern inference infrastructure and demonstrative cases. The goal remains to foster a sustained, cross-institutional "community of practice" bridging global academic and HPC centers. In addition the Inference for Science Services (INFSCI) has been built to publish activities related to the BoF community. https://infsci.nchc.org.tw/

Back to Birds of a Feather Archive Listing