Research and ACM SRC Posters Archive

CoEncode: Reclaiming Idle Encoder GPUs in Multi-Tenant Multimodal LLM Serving


Poster Type: Research Posters

Author: Mona Moghadampanah (Virginia Tech), Xiangchen Li (Virginia Tech), Cheol-Ho Hong (Chung-Ang University), Dimitrios S. Nikolopoulos (Virginia Tech)

Abstract: Disaggregated multimodal LLM serving improves stage isolation but can leave dedicated encoder GPUs substantially underutilized. We present CoEncode, a workload-aware encoder pooling approach that uses model-specific capacity profiles to provision fractional GPU resources and colocate encoder workloads while accounting for end-to-end pipeline conditions. Our evaluation shows that CoEncode maintains comparable end-to-end TTFT SLO attainment while improving SLO goodput perGPU by 19.3% and reducing encoding energy consumption by 45.9%. These results demonstrate the potential of encoder pooling to improve resource and energy efficiency without sacrificing end-to-end serving quality.

Best Poster Finalist (BP): no
Poster: PDF
Poster Summary: PDF


Back to Poster Archive Listing