Scholarlink

<p>Data heterogeneity in federated learning (FL) presents substantial challenges, manifested in various forms such as non-independent and non-identically distributed (Non-IID) data and class imbalance. These issues can severely degrade the performance of the global model, particularly by limiting its ability to generalize across diverse client datasets. Personalized federated learning (pFL) has emerged as a promising approach to mitigate these challenges. However, existing pFL methods tend to enhance local model performance, often neglecting the optimization of the global model. To address these limitations, we propose FedDIFFKD, a novel federated learning algorithm. In FedDIFFKD, the server aggregates features from participating clients to construct a comprehensive global feature set, which serves as a foundation for effective knowledge transfer. During subsequent training rounds, this global feature set guides local model training through a diffusion process, thereby refining local features. By integrating diffusion models with knowledge distillation (KD), FedDIFFKD simultaneously enhances both global and personalized models, achieving improvements in both global and local model accuracy. To further enhance the learning process, FedDIFFKD incorporates self-distillation, where knowledge from previous local models informs current iterations. FedDIFFKD eliminates the dependence on server-side public datasets or generative models, enabling it to adapt effectively to diverse federated learning scenarios. Extensive experiments on various visual datasets demonstrate that FedDIFFKD significantly outperforms existing federated learning techniques, achieving substantial improvements in both global and personalized model performance in heterogeneous data settings. The code is available online at <a href="https://github.com/ST0427/FedDIFFKD">https://github.com/ST0427/FedDIFFKD</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FedDIFFKD: integrating diffusion distillation and self-distillation for personalized federated learning

  • 期刊论文

  • 发表日期：

    2025年6月26日

查看全文
  • Junnan Liu,
  • Tao Song,
  • Rui Zhai,
  • Peng Qian,
  • Shiqiang Xu,
  • Junyang Yu

摘要

Data heterogeneity in federated learning (FL) presents substantial challenges, manifested in various forms such as non-independent and non-identically distributed (Non-IID) data and class imbalance. These issues can severely degrade the performance of the global model, particularly by limiting its ability to generalize across diverse client datasets. Personalized federated learning (pFL) has emerged as a promising approach to mitigate these challenges. However, existing pFL methods tend to enhance local model performance, often neglecting the optimization of the global model. To address these limitations, we propose FedDIFFKD, a novel federated learning algorithm. In FedDIFFKD, the server aggregates features from participating clients to construct a comprehensive global feature set, which serves as a foundation for effective knowledge transfer. During subsequent training rounds, this global feature set guides local model training through a diffusion process, thereby refining local features. By integrating diffusion models with knowledge distillation (KD), FedDIFFKD simultaneously enhances both global and personalized models, achieving improvements in both global and local model accuracy. To further enhance the learning process, FedDIFFKD incorporates self-distillation, where knowledge from previous local models informs current iterations. FedDIFFKD eliminates the dependence on server-side public datasets or generative models, enabling it to adapt effectively to diverse federated learning scenarios. Extensive experiments on various visual datasets demonstrate that FedDIFFKD significantly outperforms existing federated learning techniques, achieving substantial improvements in both global and personalized model performance in heterogeneous data settings. The code is available online at https://github.com/ST0427/FedDIFFKD.