Workshops Archive

N-VEG: 3D Gaussian Encoding for Neural Volume Compression


Workshop: ISAV26: In Situ AI, Analysis and Visualization

Authors: Eero Dunham (University of Illinois Urbana Champaign, Argonne National Laboratory); Qi Wu (NVIDIA); Landon Dyken (University of Illinois at Chicago); and Victor Mateevitsi, Silvio Rizzi, Joseph Insley, Mengjiao Han, Janet Knowles, and Michael E. Papka (Argonne National Laboratory)

Abstract: Research in Implicit Neural Representations has led to new lines of compression methods for large 3D volumetric data, offering aggressive compression ratios and direct data lookup without decompression. While these methods perform well at compression, they struggle to learn high-frequency information; though this is addressable by a variety of encoding methods, they add significant memory overhead, hampering compression goals. Other methods inspired by novel view synthesis attempt to model volumes explicitly with primitives, e.g. 3D Gaussian ellipsoids. In this work, we present a hybrid approach to volume compression that learns in-tandem a 3D Gaussian reconstruction and Neural representation of the input scene. We treat the 3D gaussians as an encoder for a small neural network, with latent feature vectors attached to each primitive. When sampling the volume, spatial locations are weighted by overlapping gaussians; feature vectors are then interpolated, harmonically activated, summed, and decoded by an MLP. Here, the 3D Gaussian encoder is a lagrangian alternative to global encoding structures, like hash grids, allowing the INR to be detail-aware when learning its encoding. This strategy achieves greater reconstruction quality than primitive-only reconstruction methods, while retaining fast training times and aggressive compression ratios, making it a viable in-situ volume compression technique.


Back to ISAV26: In Situ AI, Analysis and Visualization Archive Listing

Back to Full Workshop Archive Listing