Data-free adaptive structured pruning for federated learning

Federated learning faces challenges in real-world deployment scenarios due to limited client resources and the problem of stragglers caused by high heterogeneity. Despite efforts to reduce the training and communication overhead of federated learning through model pruning, a uniform pruning ratio fundamentally fails to address the efficiency impact of stragglers in heterogeneous systems. Therefore, adapting the pruned sub-models to individual device capabilities is crucial yet remains under-researched. In this work, we propose AdaPruneFL, a data-free adaptive structured pruning algorithm, which formulates the adaptive pruning problem in federated learning as an optimization problem constrained by aligning the response latency of the client’s local training, to identify an adaptive fine-grained model compression ratio. Combining sequential structured pruning, we extract heterogeneous but aggregable sub-model structures based on the capabilities of client devices, achieving training acceleration in a hardware-friendly manner while mitigating the straggler effect. Our extensive experiments demonstrate that, compared to FedAvg, AdaPruneFL achieves 1.38–3.88x faster training on general-purpose hardware platforms while maintaining comparable convergence accuracy.