Federated Learning (FL) enables healthcare providers to collaboratively train machine learning models while safeguarding sensitive patient data. In critical medical applications, FL facilitates leveraging multicentric datasets to improve predictive performance without compromising privacy. However, ensuring consistency and interpretability of model explanations across different institutions remains a significant challenge. Standard explainability methods like SHapley Additive exPlanations (SHAP) often yield inconsistent insights due to heterogeneity in local data distributions and isolated background datasets used during explanation generation. To address this challenge, we present SHAP–FL, a novel framework designed to harmonize SHAP-based explanations across FL clients while maintaining data privacy. To this end, our approach comprises three steps: (1) Federated Training of the shared model, (2) histogram-based Background Dataset Synthesis facilitating secure aggregation, and (3) client-sided generation of Local Explanations utilizing the previously synthesized dataset. We evaluate SHAP–FL on the task of sepsis onset prediction, a critical challenge in medical decision-making. Our results demonstrate improved accuracy and fidelity of explanations across most clients, with the generated explanations aligning more closely with established medical knowledge on sepsis.