Workshop: 2026 International Workshop on Performance, Portability, and Productivity in HPC
Authors: Burkhard Ringlein, Jan van Lunteren, Radu Stoica, and Thomas Parnell (IBM Research)
Abstract: Attention kernels represent the backbone of modern LLMs and are usually the most performance-critical GPU kernels. Today, state-of-the-art LLM inference relies primarily on attention kernels painstakingly hand-optimized for each GPU architecture. This approach delivers strong performance, but it also fragments the software ecosystem and requires substantial, ongoing engineering effort for every new model and hardware platform.
In this work, we present a case study on the performance portability of the paged attention kernel. We describe our platform-agnostic Triton Attention implementation and demonstrate that open-source domain-specific languages can deliver efficient, portable LLM inference across GPU vendors.
Our experimental evaluation shows that our Triton kernel matches the performance of platform-specific kernels on the latest NVIDIA and AMD GPUs and is up to 7x faster than existing portable attention implementations. We further discuss how our kernel improves the flexibility, portability, and developer productivity of vLLM, and share the design principles behind our approach.
Back to 2026 International Workshop on Performance, Portability, and Productivity in HPC Archive Listing
Back to Full Workshop Archive Listing