Workshop: Scale to See: Scalable AI for Scientific Imaging and Spatiotemporal Data
Authors: Chongyu Qu, Zhengyi Lu, Yanfan Zhu, Junchao Zhu, Yuechen Yang, and Juming Xiong (Vanderbilt University); Bennett Landman (Vanderbilt University, Vanderbilt University Medical Center); Yucheng Tang (NVIDIA); and Yuankai Huo (Vanderbilt University)
Abstract: The sensitivity of patient data limits the direct deployment of web-based generative AI and other cloud-hosted models in clinical environments, motivating secure, on-premise inference. However, local deployment is constrained by computational resources, memory capacity, and hardware portability, particularly for demanding tasks such as 3D medical image segmentation. In this work, we investigate low-bit post-training quantization (PTQ) for efficient offline deployment on the NVIDIA DGX Spark, a compact system with a Grace-Blackwell accelerator and 128 GB of unified memory. Using a unified TensorRT quantization workflow, we deploy nine representative 3D segmentation models spanning CNN- and transformer-based architectures, from 5.7M to 1.46B parameters, and evaluate them on the BTCV abdominal CT benchmark. Eight of nine models can be quantized to 8-bit weights and activations (W8A8) without meaningful accuracy degradation. Compared with full-precision inference, W8A8 achieves 3.9x-7.2x higher inference speed while substantially reducing memory consumption, enabling efficient, portable, privacy-preserving 3D medical image inference.
Back to Scale to See: Scalable AI for Scientific Imaging and Spatiotemporal Data Archive Listing
Back to Full Workshop Archive Listing