Workshop: PDSW’26: The 11th International Parallel Data Systems Workshop
Authors: Sitian Chen, Haonan Wang, and Amelie Chi Zhou (Hong Kong Baptist University)
Abstract: Mixture-of-Experts (MoE) inference presents system challenges because experts receive uneven workloads and exhibit different execution characteristics. This work explores how processing-in-memory (PIM) can complement GPUs to improve MoE inference efficiency. We propose a PIM-aware approach that selectively assigns suitable expert computation to PIM while retaining other computation on the GPU. The approach also considers how experts can be decomposed and mapped to available PIM resources. This WIP talk will briefly introduce the motivation and proposed solution, highlight the main design considerations, and outline the next steps toward evaluating the feasibility and potential benefits of PIM-assisted MoE inference.
Back to PDSW’26: The 11th International Parallel Data Systems Workshop Archive Listing
Back to Full Workshop Archive Listing