Workshop: The 9th Annual Parallel Applications Workshop, Alternatives to MPI+X (PAW-ATM)
Authors: Aditya Bhosale (University of Illinois Urbana-Champaign); Anant Jain and Shourya Goel (Indian Institute of Technology, Roorkee); Ritvik Rao (University of Illinois at Urbana-Champaign); Sateesh Kumar Peddoju (Indian Institute of Technology, Roorkee); and Laxmikant Kale (University of Illinois at Urbana-Champaign)
Abstract: Overdecomposition enables asynchronous many-task runtimes such as Charm++ to overlap communication with computation and to balance load dynamically. Its benefits are well established on CPUs, but GPUs favor large kernels and bulk transfers, so fine-grained execution risks prohibitive launch, synchronization, and communication overheads. We investigate whether overdecomposition can be sustained efficiently on GPUs and portably across vendors. We find that the dominant costs arise not on the device but from host-side serialization, in vendor-specific ways, and can be mitigated transparently by the runtime: by parallelizing kernel submission and completion detection across processes and threads sharing each GPU, and by routing every message through device-to-device paths. In a single-source Charm++/Kokkos environment running unchanged on NVIDIA and AMD systems, these techniques reduce the minimum effective task granularity on Task Bench from 26-37μs to 4μs, and overdecomposed Jacobi2D, MiniMD, and LULESH match MPI performance on up to 64 A40 and 32 MI250X GPUs.
Back to The 9th Annual Parallel Applications Workshop, Alternatives to MPI+X (PAW-ATM) Archive Listing
Back to Full Workshop Archive Listing