Workshops Archive

Exploiting the Interplay of Compute- and Memory-Bound kernels in MPI Applications


Workshop: PMBS26: The 17th International Workshop on Performance Modeling, Benchmarking, and Simulation of High Performance Computer Systems

Authors: Ayesha Afzal (Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen National High Performance Computing Center); Krishna Manda (University of Bonn); and Georg Hager (Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen National High Performance Computing Center)

Abstract: Parallel applications are typically designed for synchronous, lock-step execution, treating communication stalls as performance hazards. Yet, in a communication-light application with infrequent synchronization that alternates between compute-bound and memory-bound phases, an MPI communication stall can unintentionally relieve memory-bandwidth contention. We demonstrate this using a Parallel Optical Flow Solver combining a compute-bound Ray Tracing kernel with a memory-bound Optical Flow kernel and negligible inter-process communication. Natural desynchronization and overlap between the two phases yield substantial speedup, with peak performance when the number of processes concurrently executing the memory-bound phase per ccNUMA domain remains near the memory-bandwidth saturation point. We further show that MPI asynchronous progress can significantly degrade performance by increasing concurrent memory-bandwidth contention. To study the underlying mechanisms, we develop a tunable dual-kernel microbenchmark, showing that substantial application or system noise is required to achieve full desynchronization. Finally, we validate these findings with a bandwidth-aware, model-based simulator.


Back to PMBS26: The 17th International Workshop on Performance Modeling, Benchmarking, and Simulation of High Performance Computer Systems Archive Listing

Back to Full Workshop Archive Listing