Technical Papers Archive

CoreFlow: Context-Centric Resource Management for LLM-Based Agentic Workflows


Authors: Jianxiong Liao and Tiannan Fu (Sun Yat-sen University), Ruibo Fan and Zhenheng Tang (The Hong Kong University of Science and Technology), Zhi Zhou and Xu Chen (Sun Yat-sen University), and Xiaowen Chu (The Hong Kong University of Science and Technology)

Abstract: The rise of agentic workflows, which coordinate multiple LLM-based agents to tackle complex tasks, has shifted the LLM serving toward a multi-step, context-driven execution paradigm. This shift introduces highly skewed context-length distributions and agent-specific KV cache-reuse patterns in LLM workloads, challenging existing resource management systems that remain oblivious to these characteristics.

This paper introduces CoreFlow, a context-centric resource management system tailored for the distinct execution patterns of agentic workflows. CoreFlow incorporates three key designs: (1) hierarchical performance modeling, which captures the effects of context-length variance and imbalanced cache reuse inherent in context-driven execution; (2) dual-level resource optimization, combining workflow-level allocation to distribute resources across agents with agent-level partitioning to isolate requests with heterogeneous context lengths; and (3) adaptive cache management, which aligns cache placement with resource partitioning while minimizing cache-swap overhead. Evaluations on representative workflows demonstrate that CoreFlow reduces end-to-end latency by 57.6% and improves workflow throughput by 63.1%.



Presentation: file

Back to Technical Papers Archive Listing