Workshops Archive

RDMA control over lossy Ethernet: A new approach using synchronized clocks


Workshop: Workshop on High Performance Fabrics for AI and HPC

Authors: Kamran Ahmed, Akram Sbaih, and Deepak Merugu (Clockwork Systems); Anirudh Sivaraman (New York University); and Balaji Prabhakar (Stanford University)

Abstract: Remote Direct Memory Access (RDMA) underpins storage and increasingly AI systems, reducing transfer latency and host CPU overhead. Traditional deployments rely on lossless fabrics, using priority flow control, credit-based flow control, congestion notification, and substantial switch memory for skid buffers. Can RDMA operate over lossy Ethernet at a single priority without in-network enhancements? Rather than implementing sophisticated loss recovery on RDMA-capable NICs, we explore end-to-end congestion control driven by one-way delay measurements. Pervasive software clock synchronization enables senders and receivers to estimate path congestion by subtracting send timestamps from receive timestamps and use these estimates to pace transmissions. Evaluations demonstrate sub-microsecond delay measurement accuracy, enabling fine-grained RDMA control. RDMA achieves near-zero packet loss and queueing delay over lossy Ethernet while coexisting harmoniously with legacy TCP traffic. These results establish delay-based, end-to-end congestion detection and mitigation as a compelling alternative to conventional in-network mechanisms for supporting RDMA on shared Ethernet fabrics.


Back to Workshop on High Performance Fabrics for AI and HPC Archive Listing

Back to Full Workshop Archive Listing