Workshops Archive

Reducing Confusion and Broadcasting Understanding: An Empirical Study of the Semantics of NCCL


Workshop: 10th International Workshop on Software Correctness for HPC Applications (Correctness '26)

Authors: Max Stupple, Alastair Donaldson, and Azalea Raad (Imperial College London)

Abstract: We present a foundation for formalising the semantics of NCCL, the NVIDIA Collective Communications Library. NCCL is a key component in distributed machine learning and HPC stacks that utilise NVIDIA GPUs. By combining analysis of the official documentation with empirical testing of the library, we present several key insights that characterise the concurrent programming model of NCCL. This includes the introduction of the concept of command matching, and novel explanations of the conditions under which undefined behaviour may be observed.


Back to 10th International Workshop on Software Correctness for HPC Applications (Correctness '26) Archive Listing

Back to Full Workshop Archive Listing