One-shot Federated Learning (FL) has attracted tremendous attention for training a global model across decentralized devices in a single communication round. Despite the low communication cost, the efficiency of one-shot FL methods is often compromised by class imbalance conditions among clients, where the client models trained on skewed local distribution can mislead the global model. Meanwhile, auxiliary public data on the server is commonly used to distill client predictions. However, domain discrepancies between server and client data can hinder federated distillation, leading to suboptimal results. To address these issues, we propose OFedCD: One-Shot Federated Learning via Class-Aware Distillation. OFedCD treats each local data as a distinct domain and introduces a novel weighting strategy based on domain distances to aggregate reliable predictions. To enhance knowledge transfer efficiency, OFedCD employs bi-directional ensemble distillation, which aligns the global model with client prediction structures, including both inter-class and intra-class levels. We demonstrate the superiority of our method through domain adaptation generalization bound analysis and extensive experiments on benchmark datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

No Fear of Domain Discrepancy: One-Shot Federated Learning via Class-Aware Distillation

  • Siyuan Wu,
  • Yipeng Zhou,
  • Miao Hu,
  • Di Wu

摘要

One-shot Federated Learning (FL) has attracted tremendous attention for training a global model across decentralized devices in a single communication round. Despite the low communication cost, the efficiency of one-shot FL methods is often compromised by class imbalance conditions among clients, where the client models trained on skewed local distribution can mislead the global model. Meanwhile, auxiliary public data on the server is commonly used to distill client predictions. However, domain discrepancies between server and client data can hinder federated distillation, leading to suboptimal results. To address these issues, we propose OFedCD: One-Shot Federated Learning via Class-Aware Distillation. OFedCD treats each local data as a distinct domain and introduces a novel weighting strategy based on domain distances to aggregate reliable predictions. To enhance knowledge transfer efficiency, OFedCD employs bi-directional ensemble distillation, which aligns the global model with client prediction structures, including both inter-class and intra-class levels. We demonstrate the superiority of our method through domain adaptation generalization bound analysis and extensive experiments on benchmark datasets.