FedDCS: the role of sign and basic sparsity properties in federated learning
摘要
Federated learning is an emerging distributed computing technology used to prevent privacy data breaches. The focus of traditional federated learning is to improve the performance of global/personalized local models, but the fundamental reasons for performance improvement have not been studied, and which factors will affect the performance of the model. We consider the fundamental reasons for performance improvement from the perspective of model parameters, which are both sign attributes and model parameter attributes. Therefore, this article mainly studies the fundamental reasons for the improvement of model performance in federated learning from the perspectives of signs and model parameter attributes. We propose the Federated Double Constrained Space Algorithm (FedDCS), which is the first federated learning algorithm to comprehensively consider the direction and size of local model updates. This algorithm uses both sign constraints and model parameter redundancy constraints to screen local model parameters. In addition, we also introduced the concept of basic sparse attributes in the field of federated learning and provided the basic sparse attributes in the local model parameters. This is the first work in the field of federated learning to study this attribute, and studying basic sparse attributes has practical significance for the deployment of federated large models. In addition, the research content of this article is orthogonal to all current federated learning algorithms. The experimental results show that the FedDCS algorithm performs better than the baseline fedavg, fedprox, scaffled, fednova, and fed