FedDue: Optimizing Personalized Federated Learning Through Dynamic Update Classifier
摘要
In the current research on Federated Learning, the issue of heterogeneous data has become a key topic. Some methods attempt to decouple neural network into feature extraction component and classifier component and apply different processing strategies to them. The feature extraction component is designed to learn global knowledge as much as possible, while the classifier component is treated in a personalized manner. However, when the classifier is highly personalized, it will increase the risk of overfitting in the trained model, and the performance of the trained model is found to be poor when tested with low heterogeneity or homogeneous data sets. To address this issue, we propose a novel method, called \(\underline{Fed}erated~{L}earning~\underline{D}ynamic~\underline{U}pdate~{C}lassifi\underline{e}r\) (FedDue). This method dynamically adjusts the participation ratio of the global classifier in the local update, allowing the classifier to learn knowledge from other clients while maintaining its own characteristics. This dynamic control mechanism makes the model more robust and adaptable to the features and distributions of different client data. Ultimately, through extensive experiments with datasets of different types, heterogeneity levels, and distributions, we demonstrate the effectiveness of the FedDue method. This method not only significantly reduces the risk of model overfitting but also improves the model’s generalization and performance under various conditions.