In the field of Federated Learning (FL), the differences in client-side data distribution pose a challenge of poor generalization of local models. Employing knowledge distillation methods allows the transmission of complex knowledge, referred to as dark knowledge, to client models. The global features within dark knowledge can assist clients in learning out-of-distribution knowledge, thereby enhancing the generalization ability of the federated system. To tackle issues caused by the difference in data distribution, we propose a Federated Knowledge Distillation approach that leverages a prompt generator to match data distribution in FL. Firstly, we develop a personalized visual prompt generator for image classification tasks. By implicitly handling the client’s local data and training local models, this generator can effectively release data problems in distributed learning. Secondly, we utilize knowledge distillation to leverage the server’s prediction probabilities, aiding clients in acquiring global out-of-self-distribution knowledge. Experiment results on three datasets with different data distributions across different sites demonstrate a significant improvement in the classification accuracy of FL models using this approach.