Workshops Archive

AAFLOW+: Stateful Operator Abstraction with Distributed KV Cache Orchestration for Multi-Agent Workflows


Workshop: Workshop on High Performance Fabrics for AI and HPC

Authors: Arup Kumar Sarker (University of Virginia, Biocomplexity Institute and Initiative); Alexander James Halpern and Mills Staylor (University of Virginia); Aymen Alsaadi (Rutgers University); Gregor Laszewski (Biocomplexity Institute and Initiative); Shantenu Jha (Rutgers University, Princeton Plasma Physics Laboratory); Yue Cheng (University of Virginia); and Geoffrey Fox (University of Virginia, Biocomplexity Institute and Initiative)

Abstract: Multi-agent LLM workflows are distributed GPU-cluster applications that interleave retrieval, planning, and reasoning. However, text-centric agent communication repeatedly rebuilds shared context through expensive prefill, while the resulting KV cache remains confined to local serving runtimes rather than being reused as transferable state. We present AAFLOW+, a stateful abstraction of agentic workflow operators that elevates KV cache to a first-class distributed memory object, with explicit operators for materialization, transfer, fork, composition, and eviction over zero-copy, communication-aware execution graphs. Its cost-based scheduler selects transfer/recomputation using measured bandwidth and prefill costs. Evaluation uses a trace-driven model parameterized by A100/InfiniBand microbenchmarks with Mistral-7B and Llama-3-8B. Modeled results show up to $50.2\times$ TTFT reduction at long context, up to $7.63\times$ lower aggregate multi-agent compute cost at 16-agent scale, and $1.72-6.10\times$ lower peak KV memory. Transfer–recompute analysis shows KV transfer dominates recomputation at 25 Gbps and above, placing state sharing squarely within reach of HPC interconnects.


Back to Workshop on High Performance Fabrics for AI and HPC Archive Listing

Back to Full Workshop Archive Listing