Workshop: 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC)
Authors: Wyatt Spear, Luke Peyralans, and Sameer Shende (University of Oregon; ParaTools, Inc.); Ethan Sloan (University of Oregon); and Michael Heroux (ParaTools, Inc.)
Abstract: Modern HPC-AI software depends on complex webs of applications, libraries, and accelerator toolchains across many hardware platforms. E4S addresses dependency conflict resolution by distributing Spack-provisioned, containerized HPC-AI stacks across x86\_64, AArch64, and ppc64le, with CPU, CUDA, ROCm, and oneAPI variants. However, building, testing, and keeping stacks updated presents major challenges for small teams. We describe the GitLab CI pipeline and E4S Testsuite built to automate this: staged binary caching, Spack-driven environment builds, Docker/Singularity image generation, and per-package smoke tests. As of E4S 26.06, this approach drives about 200 CI jobs per pipeline, builds up to 127 CPU packages per architecture, and exercises 236 tests, with full x86\_64 CPU builds completing in 5 to 8 hours, and in minutes when cached. Automation has reduced maintainer workload, improved reproducibility, and lowered barriers to external contributions, demonstrating that CI automation is instrumental to sustaining community-maintained HPC-AI ecosystems at scale.
Back to 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC) Archive Listing
Back to Full Workshop Archive Listing