Workshops Archive

SuperSAM: Crafting a SAM Supernetwork via Structured Pruning and Unstructured Parameter Prioritization


Workshop: Scale to See: Scalable AI for Scientific Imaging and Spatiotemporal Data

Authors: Waqwoya Abebe, Sadegh Jafari, Sixing Yu, and Akash Dutta (Iowa State University); Jan Strube, Nathan R. Tallent, and Luanzheng Guo (Pacific Northwest National Laboratory); Pablo Munoz (Intel Corporation); and Ali Jannesari (Iowa State University)

Abstract: Neural Architecture Search (NAS) automates the design of efficient networks, with one-shot NAS enabling greater efficiency by training a single weight-sharing supernetwork that contains many subnetworks. Designing such search spaces is challenging due to considerations involving architectural space, scoring methods, and computational cost. We present SuperSAM, a one-shot NAS search space derived from the Segment Anything Model (SAM). Our proposed NAS approach involves automated search space design via layer-wise structured pruning and activations-based MLP parameter prioritization. The SAM supernetwork is trained with the sandwich rule, while deployment uses a program autotuner to discover efficient subnetworks. Discovered models are 30–70% smaller than the original SAM ViT-B, with some outperforming the pre-trained model.


Back to Scale to See: Scalable AI for Scientific Imaging and Spatiotemporal Data Archive Listing

Back to Full Workshop Archive Listing