FedCKD: A Knowledge Distillation Approach to Cross-Client Learning in Federated Learning with Label-Exclusive Datasets

The rapid proliferation of edge devices, such as smartphones and wearable devices, produces vast volumes of decentralized and privacy-sensitive data. Federated learning (FL) enables collaborative model training without centralizing the data, thus preserving privacy. However, in practical deployments, FL suffers significant performance degradation under label-exclusive distribution (LXD) conditions, where each client holds a private dataset with a unique, non-overlapping subset of class labels. In this paper, we propose FedCKD that enables effective cross-client collaboration through enhanced knowledge distillation (KD). We propose a novel loss function that leverages the advantages of several existing advanced KD strategies, improving knowledge transfer across clients with label-exclusive datasets. Comprehensive experiments are conducted in various data modalities, including image (CIFAR-10 and iNaturalist) and text (system logs), with various benchmark models. Experimental results show that FedCKD consistently outperforms state-of-the-art methods such as FedAvg, FedProx, FedDF, and FedVLS, demonstrating its robustness and scalability in realistic FL settings.