Workshops Archive

Towards PIM-Aware Expert Decomposition and Mapping for MoE Inference


Workshop: PDSW’26: The 11th International Parallel Data Systems Workshop

Authors: Sitian Chen, Haonan Wang, and Amelie Chi Zhou (Hong Kong Baptist University)

Abstract: Mixture-of-Experts (MoE) inference presents system challenges because experts receive uneven workloads and exhibit different execution characteristics. This work explores how processing-in-memory (PIM) can complement GPUs to improve MoE inference efficiency. We propose a PIM-aware approach that selectively assigns suitable expert computation to PIM while retaining other computation on the GPU. The approach also considers how experts can be decomposed and mapped to available PIM resources. This WIP talk will briefly introduce the motivation and proposed solution, highlight the main design considerations, and outline the next steps toward evaluating the feasibility and potential benefits of PIM-assisted MoE inference.


Back to PDSW’26: The 11th International Parallel Data Systems Workshop Archive Listing

Back to Full Workshop Archive Listing