Workshops Archive

SPEAR: Semantic Partitioning and Edge-Adapted Routing for Distributed LLM Serving


Workshop: The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO)

Authors: Dmytro Kublytskyi, Su Wang, and Dong Chen (Colorado School of Mines)

Abstract: Edge deployment of LLMs requires multiple devices working together, since no single device matches a cloud server's range. Existing edge routers treat each query as one unit and use fixed, offline-trained policies that break down as workloads change. We present SPEAR (Semantic Partitioning and Edge-Adapted Routing), which routes work at the subtask level and keeps adapting after deployment. We split each query into subtasks, send each to an edge device by capability score and risk threshold, and fall back to a cloud model only when confidence is too low. We also update each device's LoRA parameters from cloud feedback, so edge models and the router keep improving without cloud-side retraining. On a testbed of Raspberry Pi 4s, Jetson Orins, and an RTX 4090 server, SPEAR reaches 94.7% routing accuracy, beating baselines like RouteLLM, with only 32.4 ms overhead; local adaptation raises Banking77 accuracy from 31.0% to 56.5% over five rounds.


Back to The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO) Archive Listing

Back to Full Workshop Archive Listing