Communication overhead reduction in federated learning: a review

Federated learning (FL) is a decentralized machine learning approach, where multiple entities, typically devices or edge servers, collaboratively train a shared model while keeping their training data locally. This enables these entities to train the model on their local datasets and then exchange just the model updates with a central server. While this approach enhances privacy, it also introduces a communication overhead. This overhead arises from continuous updates of both the global model by the clients and the local model by the central server, referred to as update rounds. This review explores methods that mitigate the communication overhead at the data and model level, classifying them into broad categories based on their shared characteristics: communication round reduction and compression techniques. The contribution of this review lies in providing an overview of these techniques, classifying the strategies, and exploring how they can be combined for maximum communication enhancement, highlighting the factors contributing to communication load and their corresponding reduction methods. Furthermore, the review conducts a significant statistical analysis of the frequently used ML models, comparative approaches for evaluation, and datasets. Then, it considers their non-identically and independently distributed data aspects. This comprehensive analysis aims to standardize the application of these models and datasets throughout de facto. Finally, a guideline framework is provided to help researchers effectively address the communication overhead in FL.