Traditional Federated Learning (FL) enables collaborative training of deep neural networks (DNNs) across massive edge devices while preserving data privacy. However, its reliance on a centralized parameter server (PS) introduces communication bottlenecks and security risks. To address these issues, Decentralized Federated Learning (DFL) has emerged, which adopts peer-to-peer (P2P) communication to eliminate the PS. Despite its advantages, DFL faces critical challenges: limited bandwidth resources, dynamic network conditions, and heterogeneous data properties. To conquer these challenges, we design and implement a co mmunication-efficient DFL framework with peer and layer selection (MPLS), which has the following advantages. 1) Different from exchanging an entire model between two workers in previous works, each worker just collects multiple sub-models (i.e., some critical layers) from the chosen peers and stacks them into one model for aggregation. 2) MPLS adopts asynchronous training among workers without any coordinator and enables each worker to develop the peer and layer selection strategy adaptively via the proposed list scheduling algorithm. We implement MPLS on a physical platform, and extensive experiments on real-world DNNs and datasets demonstrate that MPLS achieves 2.1–4.2 \(\times \) speedup compared to the baselines.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MPLS: Stacking Diverse Layers Into One Model for Decentralized Federated Learning

  • Yang Xu,
  • Zhiwei Yao,
  • Hongli Xu,
  • Yunming Liao,
  • Zuan Xie

摘要

Traditional Federated Learning (FL) enables collaborative training of deep neural networks (DNNs) across massive edge devices while preserving data privacy. However, its reliance on a centralized parameter server (PS) introduces communication bottlenecks and security risks. To address these issues, Decentralized Federated Learning (DFL) has emerged, which adopts peer-to-peer (P2P) communication to eliminate the PS. Despite its advantages, DFL faces critical challenges: limited bandwidth resources, dynamic network conditions, and heterogeneous data properties. To conquer these challenges, we design and implement a co mmunication-efficient DFL framework with peer and layer selection (MPLS), which has the following advantages. 1) Different from exchanging an entire model between two workers in previous works, each worker just collects multiple sub-models (i.e., some critical layers) from the chosen peers and stacks them into one model for aggregation. 2) MPLS adopts asynchronous training among workers without any coordinator and enables each worker to develop the peer and layer selection strategy adaptively via the proposed list scheduling algorithm. We implement MPLS on a physical platform, and extensive experiments on real-world DNNs and datasets demonstrate that MPLS achieves 2.1–4.2 \(\times \) speedup compared to the baselines.