In federated learning, the heterogeneity of client data has a huge impact on the performance of model training. Many heterogeneous problems in this process are caused by non-IID data. Various methods have been proposed to solve the non-IID problem, and most of the previous work focuses on the difference in label shift or client shift. This study focuses on the feature shift of non-IID. In this paper, we propose a FL framework FedBC, to address this problem. FedBC retains a local batch normalization (BN) layer for each client to alleviate the feature offset, thereby preventing the global model from being negatively affected by non-IID data during the aggregation process. Meanwhile, a convolutional block attention module (CBAM) is integrated after the BN layer, and the channel and spatial attention mechanism of CBAM is used to dynamically emphasize task-related features to supplement the normalization effect of local BN. Experimental results on five benchmark datasets and three real-world datasets show that the performance of the proposed FedBC significantly outperforms the state-of-the-art FL framework by up to 8.47%, and the convergence speed is also improved.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FedBC: Using Localization Mechanism to Alleviate Non-IID Feature Shift in Federated Learning

  • Yiying Zhang,
  • Jiajie Sun,
  • Xiaokun Wang,
  • Suxiang Zhang,
  • Wei Li

摘要

In federated learning, the heterogeneity of client data has a huge impact on the performance of model training. Many heterogeneous problems in this process are caused by non-IID data. Various methods have been proposed to solve the non-IID problem, and most of the previous work focuses on the difference in label shift or client shift. This study focuses on the feature shift of non-IID. In this paper, we propose a FL framework FedBC, to address this problem. FedBC retains a local batch normalization (BN) layer for each client to alleviate the feature offset, thereby preventing the global model from being negatively affected by non-IID data during the aggregation process. Meanwhile, a convolutional block attention module (CBAM) is integrated after the BN layer, and the channel and spatial attention mechanism of CBAM is used to dynamically emphasize task-related features to supplement the normalization effect of local BN. Experimental results on five benchmark datasets and three real-world datasets show that the performance of the proposed FedBC significantly outperforms the state-of-the-art FL framework by up to 8.47%, and the convergence speed is also improved.