cFedLoRA: Clustered Aggregation for Federated LoRA
摘要
Low-Rank Adaptation (LoRA) enables the deployment of large pre-trained models in Federated Learning (FL) by updating and communicating only lightweight adapter matrices. However, training separate LoRA adapters on non-IID client data often leads to overfitting and poor cross-client generalization. We propose cFedLoRA, a clustered aggregation framework that addresses these challenges. cFedLoRA groups clients based on the similarity of their LoRA updates and aggregates local updates within each cluster, enabling communication-efficient collaboration and cluster-wise specialization. Experiments on federated benchmarks with diverse non-IID settings show that cFedLoRA achieves higher accuracy, faster convergence, and lower computational and communication costs. These improvements underscore cFedLoRA’s effectiveness and practicality for resource-constrained FL deployments.