Workshop: IA^3 2026 — 16th Workshop on Irregular Applications: Architectures and Algorithms
Authors: Noel Gomillion (Texas A&M University, Lawrence Livermore National Laboratory); Lance Fletcher (Texas A&M University); Trevor Steil (Lawrence Livermore National Laboratory); Eun Jung Kim (Texas A&M University); and Roger Pearce (Texas A&M University, Lawrence Livermore National Laboratory)
Abstract: Irregular workloads, such as those stemming from Graph Neural Networks (GNNs) and Mixture-of-Experts (MoE) models, frequently induce severe incast congestion in large-scale parallel environments. Existing fabric-level congestion control mechanisms rely on reactive, link-level rate limiting that degrades overall throughput. While receiver-driven, credit-based flow control offers a proactive alternative, scaling process counts reduces memory slots per rank below usable thresholds, rendering traditional rank-based accounting untenable. To address this challenge, we introduce MailRail, an out-of-order, asynchronous messaging framework, along with a novel user-space, node-based credit flow control system. By shifting credit accounting from individual processes to shared per-node pools, our proactive, fabric-agnostic approach maintains sufficient credit availability even as process density grows, enabling processes to dynamically utilize freed credits without waiting for receiver-side redistribution. Experimental evaluations demonstrate that node-based flow control achieves an 1132× speedup over unconstrained execution at an adversarial incast ratio of 4096:1 while adding minimal overhead on uniform workloads.
Back to IA^3 2026 — 16th Workshop on Irregular Applications: Architectures and Algorithms Archive Listing
Back to Full Workshop Archive Listing