Technical Papers Archive

Scaling Spatiotemporal Generative Compression for Earth Observation on an HPC-AI-Converged Supercomputer


Authors: Jinxiao Zhang (Tsinghua University, National Supercomputing Centre in Shenzhen (NSCS)); Xiyong Wu (Institute of Data and Information Tsinghua Shenzhen International Graduate School); Runmin Dong (Sun Yat-sen University); Xihan Huang (Institute of Data and Information Tsinghua Shenzhen International Graduate School); Shenggan Cheng (National University of Singapore); Yunkai Yang (Sun Yat-sen University); Zheng Zhou, Yunpu Xu, and Zhaoyang Luo (Institute of Data and Information Tsinghua Shenzhen International Graduate School); Fan Wei, Miao Yang, and Mengxuan Chen (Tsinghua University); Fan Yang and Bin Luo (Institute of Data and Information Tsinghua Shenzhen International Graduate School); Conghui He (Shanghai AI Laboratory); and Haohuan Fu (Institute of Data and Information Tsinghua Shenzhen International Graduate School, National Supercomputing Centre in Shenzhen (NSCS))

Abstract: Earth observation constellations generate multispectral data at unprecedented rates, creating storage and distribution bottlenecks. To reduce storage and accelerate global dissemination, we propose a scalable two-stage generative compression framework validated on annual global Sentinel-2 imagery. The encoding stage exploits large-scale spatial context and temporal redundancy, while the reconstruction stage restores the multispectral latent space and uses geolocation and observation time as metadata controls to guide generation with global historical priors. To meet the computational and memory demands of training global-scale generative models, we redesign the framework for an HPC-AI-converged supercomputer. Through topology-aware hierarchical parallelization, asymmetric HBM-DDR memory placement, and asynchronous overlap execution on Armv9 and SME, our implementation achieves end-to-end training performance comparable to high-end A100 GPU platforms under the reported workload and scales to 20,480 LineShine nodes with 76% weak-scaling efficiency. It achieves over 4,700x archive compression, outperforming existing compression methods in downstream land-cover classification.


Presentation: file

Back to Technical Papers Archive Listing