Research and ACM SRC Posters Archive

Adaptive Error Quantization for Lossy Compression of Scientific Datasets


Poster Type: ACM SRC Posters

Author: Samuel Quan (Clemson University), Jon Calhoun (Clemson University)

Abstract: Large-scale scientific instruments and applications generate massive amounts of data, creating storage and I/O bottlenecks for analysis. Error-bounded lossy compression methods mitigate this, but certain unpredictable datasets make it difficult to obtain large compression ratios with prediction-based compressors without introducing a large error. This work introduces adaptive error quantization, a novel extension to the SZ3 compressor framework. Our implementation dynamically refines the prediction error of an unpredictable data point by allocating additional error quantization bins through continuous bisection of the error, improving data fidelity for hard-to-compress datasets while minimizing storage overhead. We present a comparison of our adaptive method against the baseline SZ3 compressor on the HACC particle simulation datasets, highlighting our algorithm's advantage when compressing high-entropy fields.

Best Poster Finalist (BP): no
Poster: PDF
Poster Summary: PDF


Back to Poster Archive Listing