FedDYS: Federated Learning Based on Local Regularization Against Data Heterogeneity
摘要
In recent years, federated learning (FL) has received a great deal of attention. It is a distributed machine learning (ML) framework that essentially solves the consensus problem by training a global model. Due to the heterogeneous distribution of data across clients, the local optimal solution often deviates from the global optimal solution. To tackle this heterogeneity in FL, we study the federated optimization problem involving loss functions with possibly non-smooth local regularization terms. This ensures that the optimal model for the local empirical loss is in conformity with the global empirical loss. FedDYS is based on a three-operator splitting that enables us to decouple the local empirical loss and the regularization term. It performs the proximal operation on the local regularization term to handle non-smoothness. In addition, we provide a fixed-point iteration perspective to understand certain FL algorithms. Numerical experiments based on both real and synthetic datasets effectively demonstrate the convergence of the proposed algorithm under partial client participation.