Purpose <p>Artificial intelligence is increasingly showing potential in assisting physicians with disease diagnosis due to its improved predicting performance. In addition to achieving high performance, these systems must also be efficient and explainable. In this study, we propose a novel Feature Distillation method to transfer knowledge and explainability between models.</p> Method <p>We employed our approach to transfer both knowledge and explainability from a complex VGG19 model (teacher) to a more computationally efficient MobileNet model (student). We show how our approach effectively addresses computational challenges in identifying four pulmonary statuses in chest X-ray images. We evaluated the performance and explainability achieved with varying amounts of training data.</p> Results <p>Our results indicate that the feature-based distillation method enhances classification performance and model explainability compared to traditional knowledge distillation techniques and other direct, more computationally intensive explainability approaches. Specifically, our approach achieves up to +4% improvement in both F1-Score and explainability compared to other Explainability Distillation methods when using limited training data, demonstrating its effectiveness in balancing performance and interpretability.</p> Conclusion <p>Such improvement makes our approach particularly suitable for resource-limited environments and practical applications in medical imaging.</p>