Authors: Wesley Bland, Tiago Antunes, Lars Paul Huse, Chidambaram Muthu, Adel Abouchaev, Rabib Abrar Alam, Abdullah Alperen, Alexey Andronov, Jose Anto Akkara, Vineet Badhwar, Pavan Balaji, Daniel Berkovitch, Bartosz Bogdanski, Shmeelok Chakraborty, Sungjun Cho, John Choi, James Custer, Rodrigo De Castro, Matthew Edwards, Kristian Evensen, Even Ezell, Alex Finestead, Seth Goldstein, Wei Lin Guay, Prankur Gupta, Ranwei Hu, Adam Incera, Anand Jayaraman, Prashanth Kannan, Martin Karp, Sameer Kumar, Cristian Lumezanu, Naina Kuruballi Mahesh, Cory Modlin, Dag Moxnes, Hoang Nam Nguyen, Ashay Narsale, Jaden Padua, Kirtesh Patil, Nguyen Dinh Pham, David Ramon Prados, Amin Qassoud, Ashwin Ramachandran, Pallavi Shurpali, Gregory R. Steinbrecher, John Sundharam, Vangelis Tasoulas, Fuhou Tian, Srinivas Vaidyanathan, Vimal Vasudevan, Nicolaas Viljoen, Daniel Winkelman, Yijing Zeng, Zhaoqi Zhu, Stig Arne Olsen, Gilad Goldfarb, Rajiv Krishnamurthy, Rajeev Nair, Jonas Olsson, Joseph Provine, Sreeram Ravinoothala, Shivayogi Ugaji, Hongyi Zeng, and Nairan Zhang (Meta Platforms, Inc.)
Abstract: We present HCCL, a collective communication library co-designed with Meta's MTIA 300 accelerator, the first Meta chip to integrate backend networking directly on die. MTIA 300 includes dedicated message engines (MEs) with near-memory compute (NMC) that fully offload collective execution from the compute grid, enabling near-perfect overlap between computation and communication. HCCL uses a compiled communication model in which the host generates a complete description of each collective including dependencies. We describe the control and data path architecture, topology-aware algorithm selection across MTIA 300's asymmetric scale-up and scale-out network, and optimizations for both training and inference workloads. For training, HCCL achieves up to 940 GB/s on intra-rack collectives while introducing less than 0.5% degradation to concurrent compute throughput. For inference, we leverage one-sided communication primitives that bypass the scheduling path to minimize collective latency and describe collective designs that improve compute-communication pipelining for latency-sensitive workloads.
Presentation: file