Workshop: AI on HPC: Performance Engineering, Challenges and Opportunities
Authors: Farhana Amin, Sabiha Afroz, Mona Moghadampanah, and Dimitrios S. Nikolopoulos (Virginia Tech)
Abstract: Masked diffusion language models (dLLMs) promise faster generation than autoregressive (AR) models by denoising many tokens at once, but their behavior under concurrent serving load is unmeasured. We characterize dLLM serving using LLaDA-8B-Instruct with a D2F LoRA adapter on one NVIDIA H200 GPU, evaluated on GSM8K and HumanEval. First, request difficulty is discrete: denoising-step counts fall into 11 fixed levels, and no admission-time signal predicts the level (best R² = 0.150). Second, generation budgets below ~320 tokens understate serving variance through truncation. Third, 24% of single-request wall-clock time is GPU computation; the rest is CPU-side dispatch overhead. Sharing one forward pass per denoising step amortizes this overhead, reaching 16.0× single-request throughput at batch size 16. We also derive a batch-timeout stability rule for synchronized batching under Poisson arrivals. These results show that dLLM serving needs parallelism at the level of each denoising step. Code and data are available at https://github.com/flarebrienne/Serving-Masked-Diffusion-LLMS.
Back to AI on HPC: Performance Engineering, Challenges and Opportunities Archive Listing
Back to Full Workshop Archive Listing