Brain tumors present a medical challenge, necessitating precise diagnosis and effective treatment strategies. Early detection is paramount for enhancing patients’ quality of life afflicted with brain tumors. Magnetic resonance imaging (MRI) is a powerful diagnostic modality, while convolutional neural networks (CNN) offer adept deep learning algorithms for medical image analysis. However, the complexity and opacity inherent in neural network models make understanding by humans difficult due to the use of hidden layers and the identification of intricate patterns. In this chapter, we employ explainable artificial intelligence (XAI) techniques, specifically saliency maps, to explore the workings of a multiclass CNN model for brain tumor classification. The model was trained on MRI data sourced from three widely utilized datasets, achieving an accuracy of 99% for pituitary tumors, 98% for glioma, and 95% for meningioma. By furnishing transparent and interpretable insights into the behavior of artificial intelligence models, these techniques foster increased transparency and accountability, which is particularly critical in healthcare domains. Given the profound implications and associated risks of artificial intelligence models’ decisions on people’s lives, it is essential to understand the behavior of convolutional networks, mainly how they arrive at their predictions.