Technical Papers Archive

Towards High-Performance Sparse Graph Attention on Dense Tensor Core Units


Authors: Kejie Ma, Hailong Yang, Qingxiao Sun, Yiqing Wang, Kaige Zhang, and Kelun Lei (Beihang University); Baojian Zhou (Fudan University); Jose Flich (Universidad Politécnica de Valencia); and Depei Qian (Beihang University)

Abstract: Sparse graph attention (SGA) is a critical building block of attention-based graph neural networks (GNNs). Existing approaches either optimize individual primitives (e.g., SDDMM or SpMM) in isolation or rely on block-sparse representations, which fail to map graph-sparse workloads efficiently onto dense TCUs. We propose SpG-Attention, a unified framework that bridges sparse graph attention with dense TCU computation. Specifically, SpG-Attention introduces: (1) memory-efficient SGA (ME-SGA) parallel strategies that classify attention multiplications by their reduction characteristics and choose row/column-wise parallelization accordingly; (2) a TCU-friendly condensed-sparse format and customized fused kernels with MMA-aligned tiled execution, discrete-to-contiguous tiled loading, metadata prefetch pipelining, and register-fragment alignment; and (3) input-aware runtime optimizations based on graph statistics and refined by bounded online profiling to handle structural diversity and long-tail degree distributions. Extensive experiments on real-world graphs demonstrate that SpG-Attention achieves average speedups of 2.45x over Fused3S for inference and 2.15x over FlashSparse for end-to-end training.


Presentation: file

Back to Technical Papers Archive Listing