Workshops Archive

Charliecloud: Image Build Optimization for AI Workloads


Workshop: 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC)

Authors: Dionne Bang, Angelica Loshak, and Jordan Ogas (Los Alamos National Laboratory)

Abstract: Charliecloud is a lightweight, unprivileged container runtime for HPC. Its Git-based image build cache has structural advantages over standard layer-based caches and performs competitively, as we have previously shown. Because Git cannot track all file types or the metadata to rebuild images, Charliecloud supplements it with a complete image traversal per build instruction. Since this traversal time increases with image size, we sought to reduce it. Our approach recursively watches for changes to the image with inotifywait, a commandline tool that reports file “events” via the Linux inotify interface. Because inotifywait can sometimes miss events, we add a verification step that walks new directories. Surprisingly, monitoring speeds up some instructions but slows down others. The major gains from monitor-informed updates shrink as more files change and eventually become slowdowns, because they must be added to Git individually. This motivates a hybrid strategy that captures monitoring’s benefit while minimizing its cost.


Back to 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC) Archive Listing

Back to Full Workshop Archive Listing