<p>Federated learning, an emerging distributed learning framework, has been widely used in the field of privacy-preserving machine learning. In traditional federated learning scenarios, servers typically assume that participating parties possess sufficient computational capabilities and distribute a unified model to them for training. However, in actual federated learning application scenarios, there are numerous resource-constrained devices, such as smartwatches, smart glasses, and fitness trackers, which face significant challenges in effectively participating in federated learning training due to their resource limitations. To address these challenges, we propose FedSplit, a new federated learning algorithm that combines federated learning with split learning. First, the complete model is divided into three components: the client model component, the server model component, and the classification model component, which reduces the computational load on the client. The server only sends the client model component to the client and transfers the remaining computational tasks to the server, allowing more clients to effectively participate in the training of federated learning. Resource-constrained clients can effectively participate in federated learning training. Second, to address the data heterogeneity problem commonly encountered in federated learning and reduce model drift, FedSplit introduces a hierarchical clustering algorithm. This algorithm effectively utilizes the similarity information of client-side local data, allowing the server model to be adaptively assigned to each client for training. This approach improves the generalization ability of the local model and mitigates the negative impact of data heterogeneity. FedSplit consists of two phases. In the first phase, the main server performs hierarchical clustering on the model parameters uploaded by the clients after they train the client model component locally. In the second phase, the server and clients engage in federated learning training, where the Fed server aggregates the trained client model component, server model component, and classification model component to update the global models, which are then distributed to the clients and the main server. Extensive experiments on various datasets and settings show that FedSplit not only significantly reduces the resource load on clients but also enhances the generalization ability of the model.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FedSplit: federated learning optimization algorithm for resource constrained clients

  • Hang Liu,
  • Zhiwei Tang,
  • Rui Zhai

摘要

Federated learning, an emerging distributed learning framework, has been widely used in the field of privacy-preserving machine learning. In traditional federated learning scenarios, servers typically assume that participating parties possess sufficient computational capabilities and distribute a unified model to them for training. However, in actual federated learning application scenarios, there are numerous resource-constrained devices, such as smartwatches, smart glasses, and fitness trackers, which face significant challenges in effectively participating in federated learning training due to their resource limitations. To address these challenges, we propose FedSplit, a new federated learning algorithm that combines federated learning with split learning. First, the complete model is divided into three components: the client model component, the server model component, and the classification model component, which reduces the computational load on the client. The server only sends the client model component to the client and transfers the remaining computational tasks to the server, allowing more clients to effectively participate in the training of federated learning. Resource-constrained clients can effectively participate in federated learning training. Second, to address the data heterogeneity problem commonly encountered in federated learning and reduce model drift, FedSplit introduces a hierarchical clustering algorithm. This algorithm effectively utilizes the similarity information of client-side local data, allowing the server model to be adaptively assigned to each client for training. This approach improves the generalization ability of the local model and mitigates the negative impact of data heterogeneity. FedSplit consists of two phases. In the first phase, the main server performs hierarchical clustering on the model parameters uploaded by the clients after they train the client model component locally. In the second phase, the server and clients engage in federated learning training, where the Fed server aggregates the trained client model component, server model component, and classification model component to update the global models, which are then distributed to the clients and the main server. Extensive experiments on various datasets and settings show that FedSplit not only significantly reduces the resource load on clients but also enhances the generalization ability of the model.