New fusion loss function based on knowledge generation using Gumbel-SoftMax for federated learning

Heterogeneity of data can cause drift in federated learning (FL). Data drift negatively impacts server performance, leading to decreased convergence speed and accuracy. In this paper, a new model called FedGSM for federated learning is proposed. FedGSM proposes a novel method that generates continuous distribution labels, representing new knowledge, at the output of both clients and the serve using Gumbel-SoftMax. The proposed FL model incorporates a novel fusion loss function that guides the adjustment of model weights. By combining knowledge from both the server and clients, this loss function effectively mitigates the impact of data drift, facilitating model convergence. FedGSM offers several benefits, including a decrease in the variance level difference between server and client predictions, an enhancement in the generalization of the server model, and a reduction of the adverse effects of data drift on the performance of FL. The experimental results for the server model and clients demonstrate accuracy rates of 98.85% on the MNIST dataset and 83.74% on the CIFAR10 dataset, reflecting significant performance improvements in both server and client operations. These results indicate that the proposed model surpasses other state-of-the-art methods.