The integration of Artificial Intelligence (AI) into healthcare has the potential to revolutionize clinical diagnostics and decision-making. However, the inherent complexity of AI models, particularly Deep Learning (DL) systems, often referred to as “black boxes,” raises critical concerns about transparency, explainability, and legal accountability. This paper focuses on Explainable AI (XAI) tools in the context of tumor image classification, evaluating their performance through the lens of the EU Ethics Guidelines for Trustworthy AI and the legal framework established by the AI Act and AI Liability Directive. Three XAI tools—GradCAM, Layer-wise Relevance Propagation (LRP), and Integrated Gradients (IG)—are applied to the ResNet50 neural network model to interpret its classification of MRI brain tumor images. The tools are assessed for their technical robustness, transparency, and alignment with legal requirements. The study highlights significant variability in the performance of XAI tools. Furthermore, the article examines the implications of AI explainability for legal liability, exploring the responsibilities of system owners and users under the EU’s risk-based classification of AI systems. The findings underscore the need for continued development of XAI methods to ensure both technical and legal standards are met, enabling safer and more trustworthy deployment of AI in healthcare.