Authors: Jiefeng Zhou (University of South Florida), Sheng Di (Argonne National Laboratory), Yanfei Guo (NVIDIA Corporation), Yuhao Guo (University of South Florida), Lingqi Zhang (RIKEN Center for Computational Science (R-CCS)), Kai Zhao (Florida State University), Rajeev Thakur and Franck Cappello (Argonne National Laboratory), and Jiajun Huang (University of South Florida)
Abstract: Inter-node collective communication is a dominant bottleneck in large-scale scientific simulations and distributed artificial-intelligence workloads. Prior frameworks mitigate this via error-bounded lossy compression but face fundamental limits: scalar encoding underutilizes memory bandwidth, and outliers inflate fixed-length blocks, leaving compression ratio (CR) untapped at SIMD-friendly block sizes. We present hZCCL2, a co-designed framework that optimizes SIMD encoding, homomorphic compression, and collective communication. Its HVZ compressor achieves up to 5.31x higher compression throughput than prior state-of-the-art compressors and improves CR by up to 37.6% at SIMD-friendly block sizes with outlier-aware encoding. Its homomorphic compression is up to 8.1x faster than the decompress-operate-compress workflow. Evaluated on 3,584 CPU cores, hZCCL2 reaches up to 47.2x Bcast, 23.4x Scatter, 43.8x Allreduce, and 67.6x Alltoall peak speedups over native MPI. In an image-stacking application, it achieves 12.9x speedup at a single thread and 39.5x at 112 threads while preserving the user-specified error bound.
Presentation: file