Authors: Jiangwei Xiao, Jiarui Tian, Yuzhu Wang, and Pin Liu (School of Artificial Intelligence, China University of Geosciences, Beijing) and Jinrong Jiang (Computer Network Information Center, Chinese Academy of Sciences)
Abstract: Generative recommenders improve the modeling of long-range behavioral dependencies through unified causal sequence modeling, where the hierarchical sequential transduction unit (HSTU) serves as a core sequence encoder. However, the original HSTU implementation incurs fragmented operator execution, cross-feature reductions in LayerNorm, padding overhead from dense materialization, and redundant spatial aggregation. We present HSTUFuse, a training-compatible CUDA optimization that decomposes HSTU into three stages—pointwise projection, spatial aggregation, and pointwise transformation—with fused forward and backward implementations. It further replaces LayerNorm with Dynamic Tanh, internalizes jagged-tensor addressing, and combines valid-position computation with causal lower-triangular pruning. ML-1M training runs show stable convergence while retaining recommendation quality close to the PyTorch HSTU. On ML-20M, it achieves per-batch HSTU execution speedups of 19.53× on RTX 4090 and 22.46× on H800, measured using the sum of forward- and backward-pass times.
Presentation: file