The heart, being a vital organ in the human body, requires continuous monitoring to detect various cardiovascular diseases. Electrocardiography (ECG) has been one of the oldest and most effective methods for diagnosing heart diseases, providing critical insights into the heart’s electrical activity. However, interpreting ECG signals is complex, requiring trained and expert cardiologists, leading to potential delays in diagnosis and treatment. To address this challenge, an automated diagnostic system is essential to facilitate timely and accurate detection of heart conditions. In this research, we introduce a hybrid model combining Convolutional Neural Networks (CNN) and XGBoost as one whole architecture to classify ECG images into four categories: Normal, Myocardial Infarction (MI), Abnormal, and Previous History of MI. Our model achieves an impressive accuracy of 98%, along with excellent precision, recall, and F1 scores, demonstrating its robustness in handling multi-class classification. The need for transparency in the machine and deep learning models, particularly in healthcare, is crucial for clinical acceptance. To ensure interpretability, we employ the Score-CAM (Score- Weighted Class Activation Maps) technique, which generates saliency maps to visualize the important regions of ECG images that influence the model’s decision-making process. These visualizations provide valuable insights into the model’s focus on diagnostically significant features such as the QRS complex, ST-segment elevation, and T wave patterns, thereby enhancing the trust and reliability of the model in medical applications.