Workshops Archive

Theoretical and Empirical Analysis of Sparse Convolution


Workshop: AI on HPC: Performance Engineering, Challenges and Opportunities

Authors: Keshvi Tuteja (Karlsruhe Institute of Technology); Yu-Hsiang Tsai and Gregor Olenik (Technical University of Munich); Markus Götz (Karlsruhe Institute of Technology, Technical University of Munich); Achim Streit (Karlsruhe Institute of Technology); Hartwig Anzt (Karlsruhe Institute of Technology, University of Tennessee); and Charlotte Debus (Karlsruhe Institute of Technology)

Abstract: Deep neural networks for image processing are increasingly deployed on resource-constrained edge devices such as drones, medical systems, and smartphones, where limited computational resources and memory bandwidth challenge efficient inference. Pruning often produces highly sparse convolutional kernels, offering the potential to reduce computation and memory requirements. However, exploiting this sparsity remains challenging due to irregular memory accesses and workload imbalance. In this work, we investigate standard convolution with sparse weights and analyze its performance using roofline analysis, showing that a 3 × 3 sparse convolution achieves an arithmetic intensity of ~10 FLOP/byte compared to ∼96 FLOP/byte for dense convolution. We propose techniques for efficient sparse convolution at high sparsities. Preliminary results show that our approach achieves up to 1.65× speedup over dense convolution on NVIDIA A100 and up to 1.52× on NVIDIA Tegra X1, demonstrating the potential of direct sparse convolution for efficient inference on modern GPUs and edge devices.


Back to AI on HPC: Performance Engineering, Challenges and Opportunities Archive Listing

Back to Full Workshop Archive Listing