Scholarlink

In generative artificial intelligence, many studies focus on the use of federated learning in diffusion models since such federated generative diffusion models significantly solve challenges of generative models on large centralized datasets in terms of privacy, copyright, and data authority. Nevertheless, the federated generative diffusion models involve frequent transmission of models between the server and clients, thus leading to extremely high communication cost. Besides, it remains unclear how federated learning affects diffusion models. To address these challenges, we develop a more effective denoising diffusion probabilistic model (DDPM) with lower parameters by leveraging channel-mixing multi-layer perceptrons (MLP) in U-Net, rather than utilizing the attention mechanism in all existing DDPMs. More importantly, we develop a novel method for training DDPM, coined PBFedDiff, by introducing the Powerball method in the federated aggregation, significantly accelerating the model convergence and impacting communication efficiency. Extensive experiments demonstrate that our approach consistently outperforms the baseline methods across all datasets, achieving 18.7%, 13.8%, and 12.9% lower Fréchet Inception Distance (FID) (widely used to represent image quality) on the dataset CIFAR-10 under three different dataset distributions, namely independent and identically distributed (IID), label-skew and quantity-skew, respectively.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Lightweight Diffusion Models with Federated Learning: Attention No Necessary for U-Net

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Wencong Jia,
  • Jie Meng,
  • Zhuang Yang

摘要

In generative artificial intelligence, many studies focus on the use of federated learning in diffusion models since such federated generative diffusion models significantly solve challenges of generative models on large centralized datasets in terms of privacy, copyright, and data authority. Nevertheless, the federated generative diffusion models involve frequent transmission of models between the server and clients, thus leading to extremely high communication cost. Besides, it remains unclear how federated learning affects diffusion models. To address these challenges, we develop a more effective denoising diffusion probabilistic model (DDPM) with lower parameters by leveraging channel-mixing multi-layer perceptrons (MLP) in U-Net, rather than utilizing the attention mechanism in all existing DDPMs. More importantly, we develop a novel method for training DDPM, coined PBFedDiff, by introducing the Powerball method in the federated aggregation, significantly accelerating the model convergence and impacting communication efficiency. Extensive experiments demonstrate that our approach consistently outperforms the baseline methods across all datasets, achieving 18.7%, 13.8%, and 12.9% lower Fréchet Inception Distance (FID) (widely used to represent image quality) on the dataset CIFAR-10 under three different dataset distributions, namely independent and identically distributed (IID), label-skew and quantity-skew, respectively.