Scholarlink

<p>Federated Reinforcement Learning (FRL) has emerged as a promising approach to facilitate collaborative policy learning among distributed agents while preserving data privacy. However, the susceptibility of FRL to Byzantine attacks undermines its performance and exacerbates uncertainties during the training process. In practical deployment, effectively mitigating Byzantine attacks instigated by malicious agents poses a formidable challenge. To confront this issue, we present a robust FRL framework, Fed-DQC, which integrates a novel Byzantine-resilient mechanism into Double Deep Q-Network (DDQN). The core of Fed-DQC lies in its distinctive parameter analysis and model similarity measurement mechanism. This mechanism can effectively identify and mitigate the influence of malicious agents, thereby curtailing the destructive impact of Byzantine attacks on the training process. Leveraging the distinctive nature of reinforcement learning, we propose an innovative two-stage federated aggregation scheme designed to maximize the utilization of agents’ exploration processes while minimizing the risk of misidentifying malicious agents. Our extensive experiments show that Fed-DQC exhibits strong robustness and adaptability against five different Byzantine attacks, achieving even better performance than scenarios unaffected by Byzantine attacks.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Byzantine-robust federated reinforcement learning via critical parameter analysis

  • 期刊论文

  • 发表日期：

    2025年9月8日

查看全文
  • Wenzheng Jiang,
  • Ji Wang,
  • Weidong Bao,
  • Yaohong Zhang

摘要

Federated Reinforcement Learning (FRL) has emerged as a promising approach to facilitate collaborative policy learning among distributed agents while preserving data privacy. However, the susceptibility of FRL to Byzantine attacks undermines its performance and exacerbates uncertainties during the training process. In practical deployment, effectively mitigating Byzantine attacks instigated by malicious agents poses a formidable challenge. To confront this issue, we present a robust FRL framework, Fed-DQC, which integrates a novel Byzantine-resilient mechanism into Double Deep Q-Network (DDQN). The core of Fed-DQC lies in its distinctive parameter analysis and model similarity measurement mechanism. This mechanism can effectively identify and mitigate the influence of malicious agents, thereby curtailing the destructive impact of Byzantine attacks on the training process. Leveraging the distinctive nature of reinforcement learning, we propose an innovative two-stage federated aggregation scheme designed to maximize the utilization of agents’ exploration processes while minimizing the risk of misidentifying malicious agents. Our extensive experiments show that Fed-DQC exhibits strong robustness and adaptability against five different Byzantine attacks, achieving even better performance than scenarios unaffected by Byzantine attacks.