Workshops Archive

Error-Explicit Low-Precision Factorizations for Iterative Refinement


Workshop: High-Accuracy Computing on Low-Precision Hardware

Authors: Joseph McLaughlin and Jee Choi (University of Oregon)

Abstract: Recent NVIDIA GPUs prioritize low-precision throughput while native FP64 stagnates, falling from $37$ to $1.25$\,TFLOP/s between B200 and B300. Mixed-precision iterative refinement factorizes in FP32 but retains the FP64 matrix $A$ for residual evaluation, requiring $12n^2$ bytes of persistent storage. We present two refinement algorithms that remove this dependence. Split-$A$ iterative refinement (SA-IR) represents $A$ as two FP32 words, while error-explicit iterative refinement (R-IR) stores the FP32 factorization residual $R=PA-LU$ and discards $A$, reducing storage to $8n^2$ bytes. R-IR inherits refinement's convergence condition, and in our tests its accuracy is limited by residual evaluation rather than FP32 storage of $R$. Across five GPUs the advantage tracks the FP32:FP64 ratio. On B300 at $n=32{,}768$, SA-IR is $25\times$ faster than emulated FP64 and $1.4\times$ faster than vendor refinement, and R-IR is $24\times$ faster at $8192$ right-hand sides. Streamed construction of $R$ peaks at $8.24$--$8.57n^2$ bytes, reaching $n=184{,}320$ where $12n^2$ will not fit.


Back to High-Accuracy Computing on Low-Precision Hardware Archive Listing

Back to Full Workshop Archive Listing