Workshop: 6th International Workshop on RESource DISaggregation in High Performance Computing (RESDIS)
Authors: Magnus Kalland (Dolphin ICS, Simula Research Laboratory); Jonas Markussen and Lars Kristiansen (Dolphin ICS); Carsten Griwodz (University of Oslo); and Tore Larsen, HÃ¥kon Stensland, and Tor Skeie (Simula Research Laboratory)
Abstract: Routable PCIe enables composable infrastructures and device pooling with native drivers, yet little is known about its end-to-end performance in production systems, leaving a gap between microbenchmark research and marketing whitepapers. We show routable PCIe challenges assumptions made by traditional operating systems, particularly in how DMA buffers are managed when accessed by remote devices. We resolve a cache incoherence issue in the IOMMU's interrupt remapping subsystem and adapt IOMMU scalability optimizations from the OS kernel to routable PCIe software, reducing median DMA deallocation latency by 90% versus prior work and enabling full parallelism for pooled devices. We further optimize the deallocation critical path via a novel IOMMU sidecore-offloading scheme, reducing median unmapping latency by 26% and tail latency by up to 89% compared to the kernel. Finally, we evaluate on server-grade storage and GPUs, showing routable PCIe achieves native IOPS at lower latency and CPU utilization than scale-out alternatives.
Back to 6th International Workshop on RESource DISaggregation in High Performance Computing (RESDIS) Archive Listing
Back to Full Workshop Archive Listing