Workshop: 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC)
Authors: Radostin Stoyanov (University of Oxford, Doubleword); Fergus Finn (Doubleword); Dan Feigin (NVIDIA); Andrei Vagin (CRIU); and Rodrigo Bruno (INESC-ID, IST, University of Lisbon)
Abstract: Inference services incur cold-start costs from KV-cache profiling and CUDA-graph construction. Container checkpoints avoid repeated initialization, but large memory images consume storage and serial reconstruction delays restoration. We present PROCS, a parallel, restore-optimized compression method for CRIU snapshots. PROCS stores pages in VMA-bounded, page-addressable blocks, supports partial and incremental reads, and reconstructs bounded batches using reusable workers. It coordinates parallel restoration of private mappings, memory-backed files, and anonymous shared memory under restore-wide CPU and buffer limits. We implement PROCS in CRIUgpu and evaluate five SGLang inference models. With 256 KiB blocks and serial restoration, PROCS reduces snapshot size by 12.5–45.5% and restore-to-first-token latency by 7.2–44.7% relative to uncompressed CRIUgpu. At fixed parallelism, 1 MiB blocks reduce size by 6.2%, checkpoint time by 12.4%, and restore-to-first-token latency by 22.5% relative to 4 KiB encoding. For Qwen3.5 27B FP8, increasing decompression parallelism to 16 threads reduces CRIU’s restore latency by 25.8%.
Back to 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC) Archive Listing
Back to Full Workshop Archive Listing