Workshops Archive

SplitFT: An Edge-Cloud Collaborative Federated Split Learning System for LLM Fine-Tuning


Workshop: The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO)

Authors: Yimeng Shan (Hong Kong Polytechnic University); Zhaorui Zhang (The Hong Kong Polytechnic University); Sheng Di (Argonne National Laboratory); Yu Liu (The Hong Kong Polytechnic University); Xiaoyi Lu (University of California, Merced); and Benben Liu (The University of Hong Kong)

Abstract: Federated Split Learning (FSL) has emerged as an effective paradigm for mitigating the computational burden of edge clients in classical federated learning while preserving data privacy. However, applying FSL to large language model (LLM) fine-tuning introduces several critical challenges. In particular, cut-layer selection must be adaptively configured across clients to accommodate heterogeneous data distributions and device capabilities, which can significantly affect system performance. Moreover, reducing communication overhead during fine-tuning remains an important yet underexplored problem.

We propose SplitFT, an edge-cloud collaborative federated split learning system for LLM fine-tuning. SplitFT allows clients to select different cut layers according to their computational resources and model performance. It further reduces communication costs by decreasing the LoRA rank at the cut layer. To better emulate real-world data heterogeneity, we introduce a length-based Dirichlet partitioning strategy. Extensive experiments on popular benchmarks demonstrate that SplitFT outperforms state-of-the-art methods in both fine-tuning efficiency and model performance.


Back to The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO) Archive Listing

Back to Full Workshop Archive Listing