Scholarlink

This paper introduces Federated Attention Fisher (FedAF), a new framework for privacy-preserving, decentralized model training in Federated Learning (FL). FL allows collaborative training across devices without centralizing raw data, addressing privacy and security concerns. However, FL struggles with data heterogeneity, communication costs, and security risks like model poisoning. FedAF tackles these issues by using Attention Mechanisms and Fisher Information Matrix (FIM) to improve model aggregation, communication efficiency, and robustness. Attention Mechanisms enable fine-grained, layer-wise aggregation based on parameter similarity, while Fisher Information helps extract knowledge from distributed data efficiently. Experiments on datasets like FashionMNIST, SVHN, CIFAR10, and CINIC10 show that FedAF outperforms existing FL methods in perplexity and communication rounds. Ablation studies confirm the importance of Attention Mechanisms and Fisher Information in boosting performance and efficiency. This work advances FL, enabling scalable, privacy-aware AI applications that comply with regulations and build user trust in decentralized systems.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FedAF: Federated Learning Framework Based on Attention Mechanisms and Fisher Information Matrix

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yuanhong Xiao,
  • Canghong Jin,
  • Lisha Peng

摘要

This paper introduces Federated Attention Fisher (FedAF), a new framework for privacy-preserving, decentralized model training in Federated Learning (FL). FL allows collaborative training across devices without centralizing raw data, addressing privacy and security concerns. However, FL struggles with data heterogeneity, communication costs, and security risks like model poisoning. FedAF tackles these issues by using Attention Mechanisms and Fisher Information Matrix (FIM) to improve model aggregation, communication efficiency, and robustness. Attention Mechanisms enable fine-grained, layer-wise aggregation based on parameter similarity, while Fisher Information helps extract knowledge from distributed data efficiently. Experiments on datasets like FashionMNIST, SVHN, CIFAR10, and CINIC10 show that FedAF outperforms existing FL methods in perplexity and communication rounds. Ablation studies confirm the importance of Attention Mechanisms and Fisher Information in boosting performance and efficiency. This work advances FL, enabling scalable, privacy-aware AI applications that comply with regulations and build user trust in decentralized systems.