Authors: Mausam Basnet and Tong Shu (University of North Texas)
Abstract: Over 100,000 public LoRA adapters now share a small number of base LLMs, and cloud providers increasingly serve them for independent tenants whose private data must stay isolated. Serverful systems co-locate all adapters in one process for efficient batching but sacrifice isolation; serverless systems run each adapter in an isolated function but redundantly recompute the shared base-model projection in every one, wasting GPU memory bandwidth. We present SwarmLoRA, a disaggregated serverless architecture that separates shared computation from per-adapter state. A persistent aggregator batches the base-model projection once across all active functions; isolated worker functions each hold one adapter’s weights and KV cache and scale independently. This design achieves cross-adapter batching without breaking per-adapter isolation. SwarmLoRA achieves 4.1×-4.9× higher throughput on four L40S GPUs than state-of-the-art ServerlessLLM and ServerlessLoRA, while preventing cross-tenant data leakage through malicious adapters. SwarmLoRA shows that multi-tenant LoRA serving need not trade tenant isolation for batching efficiency.
Presentation: file