In recent years, federated learning (FL) has received a great deal of attention. It is a distributed machine learning (ML) framework that essentially solves the consensus problem by training a global model. Due to the heterogeneous distribution of data across clients, the local optimal solution often deviates from the global optimal solution. To tackle this heterogeneity in FL, we study the federated optimization problem involving loss functions with possibly non-smooth local regularization terms. This ensures that the optimal model for the local empirical loss is in conformity with the global empirical loss. FedDYS is based on a three-operator splitting that enables us to decouple the local empirical loss and the regularization term. It performs the proximal operation on the local regularization term to handle non-smoothness. In addition, we provide a fixed-point iteration perspective to understand certain FL algorithms. Numerical experiments based on both real and synthetic datasets effectively demonstrate the convergence of the proposed algorithm under partial client participation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FedDYS: Federated Learning Based on Local Regularization Against Data Heterogeneity

  • Jiao Xue,
  • Chundong Wang

摘要

In recent years, federated learning (FL) has received a great deal of attention. It is a distributed machine learning (ML) framework that essentially solves the consensus problem by training a global model. Due to the heterogeneous distribution of data across clients, the local optimal solution often deviates from the global optimal solution. To tackle this heterogeneity in FL, we study the federated optimization problem involving loss functions with possibly non-smooth local regularization terms. This ensures that the optimal model for the local empirical loss is in conformity with the global empirical loss. FedDYS is based on a three-operator splitting that enables us to decouple the local empirical loss and the regularization term. It performs the proximal operation on the local regularization term to handle non-smoothness. In addition, we provide a fixed-point iteration perspective to understand certain FL algorithms. Numerical experiments based on both real and synthetic datasets effectively demonstrate the convergence of the proposed algorithm under partial client participation.