Communication-efficient federated learning based on compressed sensing and ternary quantization

Most existing work on Federated Learning (FL) transmits full-precision weights, which contain a significant amount of redundant information, leading to a substantial communication burden. This issue is particularly pronounced with the growing prevalence of smart mobile and Internet of Things (IoT) devices, where data sharing generates a large communication cost. To address this issue, we propose a communication-efficient Federated Learning algorithm, FedCSTQ, based on compressed sensing (CS) and ternary quantization.FedCSTQ introduces a heuristic sparsification method that enhances information selection, thereby mitigating the accuracy degradation typically associated with CS. Additionally, the algorithm incorporates ternary quantization to process residuals after sparsity, further reducing the impact of accuracy degradation due to sparsity while guaranteeing a small amount of communication overhead. Experiments conducted on the publicly available datasets reveal that FedCSTQ outperforms the standard FL (FedAvg), SignSGD with a majority vote, FL using dithering(CEP-FL), and FL based on Compressed Sensing (CS-FL). Ablation studies further demonstrate the effectiveness of our method.