Workshop: High-Accuracy Computing on Low-Precision Hardware
Authors: Varuni Katti Sastry (Argonne National Laboratory (ANL)) and Venkat Vishwanath (Argonne National Laboratory)
Abstract: Wafer-scale AI engines pack hundreds of thousands of cores onto a single die, but their arithmetic is low-precision, with no native double precision (FP64)—the workhorse of sci- entific computing. We emulate FP64 matrix multiply (GEMM) entirely on the wafer, to the accuracy of a true FP64 computation, using the Ozaki scheme to split each operand into bfloat16 slices. On a grid with no shared memory, the Ozaki slices must be reduced on the wafer itself in extended precision; otherwise the reduction would discard the low-order bits the split preserves. Accuracy is FP64-class at every size we measured, up to 3360 × 3360, and we scale FP64 GEMM to span∼50% of the wafer. The method extends to complex FP64 through a host- side Karatsuba wrapper and drives a Newton–Schulz solver to FP64-class residuals.
Back to High-Accuracy Computing on Low-Precision Hardware Archive Listing
Back to Full Workshop Archive Listing