Enhancing Federated Learning Performance Fairness via Collaboration Graph-Based Reinforcement Learning

Federated learning has recently developed into a pivotal distributed learning paradigm, wherein a server aggregates numerous client-trained models into a global model without accessing any client data directly. It is acknowledged that the impact of statistical heterogeneity in client local data on the pace of global model convergence, but it is often underestimated that this heterogeneity also engenders a biased global model with notable variance in accuracy across clients. Contextually, the prevalent solutions entail modifying the optimization objective. However, these solutions often overlook implicit relationships, such as the pairwise distances of site data distributions, which makes pairwise exclusive or synergistic optimization among client models. Such optimization conflicts compromise the efficacy of earlier methods, leading to performance imbalance or even negative transfer. To tackle this issue, we propose a novel aggregation strategy called Collaboration Graph-based Reinforcement Learning (FedGraphRL). By deploying a reinforcement learning (RL) agent equipped with a multi-layer adaptive graph convolutional network (AGCN) on the server-side, we can learn a collaboration graph from client state vectors, revealing the collaborative relationships among clients during optimization. Guided by an introduced reward that balances fairness and performance, the agent allocates aggregation weights, thereby promoting automated decision-making and improvements in fairness. The experimental results on two real-world multi-center medical datasets suggest the effectiveness and superiority of the proposed FedGraphRL.