Efficient federated learning with timely update dissemination
摘要
Federated learning (FL) has emerged as a compelling methodology for the management of distributed data, marked by significant advancements in recent years. This paradigm facilitates collaborative model training by utilizing raw data distributed across multiple edge devices. Compared to traditional machine learning approaches that require centralized data collection, FL offers significant advantages in terms of privacy preservation and the efficient use of edge computing resources. Nonetheless, the data are typically characterized by non-Independent and Identically Distributed properties, resulting in statistical heterogeneity that can adversely affect model accuracy. Moreover, the disparity in computational and communication capabilities among edge devices (system heterogeneity) further extends the duration of the training process. To address these challenges associated with heterogeneity, we propose an efficient FL approach that capitalizes on additional downlink bandwidth resources to ensure timely update dissemination. Initially, we implement this strategy within an asynchronous framework, introducing the Asynchronous Staleness-aware Model Update (FedASMU), which integrates both server-side and device-side methodologies. On the server side, we present an asynchronous FL system model that employs a dynamic model aggregation technique, which harmonizes local model updates with the global model to enhance both accuracy and efficiency. Concurrently, on the device side, we propose an adaptive model adjustment mechanism that integrates the latest global model with local models during training to further elevate accuracy. Subsequently, we extend this approach to a synchronous context, referred to as FedSSMU. Theoretical analyses substantiate the convergence of our proposed methodologies. Extensive experiments, encompassing six models and five public datasets, demonstrate that FedASMU and FedSSMU significantly surpass baseline methods in terms of both accuracy (up to 145.87%) and efficiency (up to 97.59%).