Workshop: IA^3 2026 — 16th Workshop on Irregular Applications: Architectures and Algorithms
Authors: Sinan Ekmekçibaşı (Koç University); James Trotter (Simula Research Laboratory); Johannes Langguth (Simula Research Laboratory, University of Bergen); Xing Cai (University of Oslo, Simula Research Laboratory); and Didem Unat (Koç University)
Abstract: We introduce a novel triple-overlap implementation for the pipelined preconditioned conjugate gradient (Pipe-PCG) and pipelined stabilized biconjugate gradient (Pipe-BiCGStab) solvers. Our approach extends conventional computation-communication overlap to concurrently execute collective operations, point-to-point communication, and compute kernels. This enables true computation-communication-communication overlap across multiple GPUs.
We evaluate on the MareNostrum 5 (H100) and Leonardo (A100) supercomputers, comparing against PETSc and the state-of-the-art two-stream NCCL implementation using SuiteSparse matrices and three massive instances scaled up to 512 GPUs. The three-stream design consistently outperforms the two-stream baseline, with the margin widening from 2–6% at 4 GPUs to 18–47% at 64 GPUs.
At 64 GPUs, the geometric-mean speedup over the two-stream baseline is 1.33× on MareNostrum 5 and 1.23× on Leonardo for Pipe-PCG, reaching 1.30× for Pipe-BiCGStab. In large-scale experiments, we observed speedups of up to 35% (Pipe-PCG) and 33% (Pipe-BiCGStab) over the two-stream design, and up to 3.27× over PETSc for both solvers.
Back to IA^3 2026 — 16th Workshop on Irregular Applications: Architectures and Algorithms Archive Listing
Back to Full Workshop Archive Listing