Workshop: High-Accuracy Computing on Low-Precision Hardware
Authors: Shun-ichiro Hayashi (Graduate School of Informatics Nagoya University) and Daichi Mukunoki, Tetsuya Hoshino, and Takahiro Katagiri (Information Technology Center Nagoya University)
Abstract: This paper proposes a method and its implementation for emulating FP64 matrix multiplication (DGEMM) by constructing, on FP4 (E2M1) Tensor Cores, Ozaki schemes I and II. The key property is that every FP4 value becomes an integer when doubled, and that shifting this integer set by multiples of 13 covers all integers. Converting an arbitrary integer into base-13 FP4 limbs keeps intermediate sums error-free in FP32 accumulators. By the same principle, the integer GEMM of INT8 Tensor Cores can be emulated bit-exactly on FP4. When FP4 Tensor Cores have twice the throughput of FP8, Ozaki scheme II theoretically achieves slightly higher performance than its FP8 counterpart. This paper further proposes kernel optimizations raising the attained fraction of peak performance. We verify this on an RTX PRO 6000 Blackwell, achieving performance competitive with an existing FP8-based implementation of Ozaki scheme II, and exceeding it at a large problem size (16384^3).
Back to High-Accuracy Computing on Low-Precision Hardware Archive Listing
Back to Full Workshop Archive Listing