Technical Papers Archive

TIDES: Tiered Block-Sparse Tensor Contraction with Streaming Transpose on GPUs


Authors: Zecheng Li, Sri Harshavardhan Reddy Deverapalli, Yanbo Zhao, and Frank Mueller (North Carolina State University); Karl Pierce (Flatiron Institute); and Jiajia Li (North Carolina State University)

Abstract: Block-sparse tensor contraction is a core kernel in symmetry-preserving many-body methods such as DMRG, where physical symmetries produce thousands of block pairs that vary widely in shape and tensor order, with element counts spanning several orders of magnitude. We present TIDES, a hybrid GPU algorithm that matches this heterogeneity with three execution tiers: a warp-level dot-product kernel for small blocks, a shared-memory direct contraction kernel for medium blocks, and, for large blocks, a streaming TTGT pipeline over grouped GEMM whose transpose workspace is bounded by the largest single block pair and whose HyperTile kernel transposes many heterogeneous high-order blocks in one grouped launch. On 2D Hubbard DMRG workloads across four symmetry regimes on an NVIDIA H100, TIDES achieves geometric-mean speedups of 2.14x (FP64) and 1.69x (FP32) over NVIDIA cuTENSOR 2.6.0, with speedups up to 5.97x and 9.86x, respectively, lower setup overhead, and portable performance on other GPUs.


Presentation: file

Back to Technical Papers Archive Listing