FedSCD: Federated Learning with Semi-centralization, Discrepancy-Awareness and Dual-Model Collaboration

Federated learning enables collaborative model training without sharing local data. However, traditional centralized frameworks impose high demands on the aggregation server. In contrast, decentralized solutions allow direct communication between clients, but it is difficult to implement some global optimization due to heterogeneous data. To balance privacy and performance, we propose FedSCD, a semi-centralized framework. FedSCD trains a private model and a low-accuracy proxy model on each client, while only proxy models are shared across clients. Local and global data discrepancies determine the weight of the two models’ losses for aggregation. Furthermore, knowledge distillation and adaptive learning rate are utilized for dual-model co-training. Experiments demonstrate that compared to state-of-the-art centralized averaging schemes, FedSCD avoids the clients directly sending high-precision private models, and relaxes the requirements of the aggregation server in terms of computing power and reliability. Compared to decentralized methods, FedSCD achieves a good balance between privacy protection and high performance. Through discrepancy-awareness adaptive training, FedSCD improves the performance of private models while limiting the growth of the accuracy of proxy models.