FedQL: Q-Learning Guided Aggregation for Federated Learning

Federated learning is a distributed machine learning paradigm, which is able to achieve model training without sharing clients’ private data. In each round, after receiving a global model, selected clients train the model with local private data and report updated parameters to server. Then the server performs aggregation to generate a new global model. Currently, aggregations are generally conducted in a heuristic manner, and show great challenges with non-Independent and Identically Distributed (non-IID) data. In this paper, we propose to employ Q-learning to solve the aggregation problem under non-IID data. Specifically, we define state, action as well as reward in the target aggregation scenario, and fit it into Q-learning framework. With the learning procedure, effective actions indicating weights assignment for aggregation can be figured out according to certain system states. Evaluation shows that the proposed FedQL strategy can improve the convergence speed obviously under non-IID data, when compared with existing schemes FedAvg, FedProx and FedAdp.