Workshops Archive

PRISM: Task-aware Slimming for Edge-Cloud LLMs


Workshop: The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO)

Authors: Su Wang, Dmytro Kublytskyi, and Dong Chen (Colorado School of Mines)

Abstract: While LLM compression techniques for cloud and HPC environments often assume diverse, unpredictable workloads, edge deployments typically serve fixed, single-task objectives. We identify two important distinctions in edge-based serving: first, unlike cloud inference which is bound by attention and KV-cache transfers, edge inference is primarily bottlenecked by MLP layers; second, layer and weight importance exhibit strong, task-specific sensitivity signatures. Building on these insights, we present PRISM, a task-aware compression framework for edge-cloud LLM serving. In cloud, PRISM profiles per-task sensitivity using causal ablation and AWQ-style quantization error, generating an execution layer mask and a precision weight mask to form a task-specialized inference DAG. On the edge, a novel sliding-chunk scheduler minimizes the resident memory footprint by streaming weights and overlapping I/O with computation. Evaluated on SmolLM2 and Qwen2.5 across four tasks, PRISM reduces peak memory by up to 2.9x and end-to-end latency by up to 26% compared to dense baselines.


Back to The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO) Archive Listing

Back to Full Workshop Archive Listing