Workshops Archive

Characterizing LLM Inference Performance on a WebAssembly Runtime


Workshop: 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC)

Authors: Aono Shigematsu (Ochanomizu University); Mari Yamaoka, Nozomi Aoki, Akira Hirai, and Masahiro Miwa (Fujitsu Limited); and Masato Oguchi (Ochanomizu University)

Abstract: WebAssembly (WASM) has drawn attention as a portable binary format and a promising substrate for AI workloads. Delegating computation to a native backend through wasi-nn promises portability with native speed, yet to our knowledge this approach has not been evaluated quantitatively for LLM inference. We compare LlamaEdge, a WASM-based LLM runtime, with native llama-cli on a multi-socket node, aligning backend, build settings, and prompts so that only the execution environment differs. Across 1B–8B models, thread counts, context lengths, and GPU offload ratios, performance is equivalent under full GPU offload, while CPU-only execution shows a gap of up to 1.38×. We attribute this gap primarily to the in-page offset of the work buffer, set by the host-side allocation path, rather than WASM overhead. Confining cores and memory to one node reduces the gap to 1.07–1.13×, and allocating the work buffer from the main heap narrows it further to 1.02–1.04×.


Back to 8th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC) Archive Listing

Back to Full Workshop Archive Listing