<p>Breast cancer is one of the most aggressive diseases, and early detection is crucial. Traditionally, machine learning has been used for detection, however, recently, convolutional neural networks (CNNs) have become prominent in developing computer-aided diagnosis systems for cancer. Most models focus on a single modality of breast images. This paper proposes a customized CNN model designed to analyze breast images from multiple modalities. The model comprises four convolutional layers and four fully connected layers, effectively classifying the disease into three classes: normal, malignant, and benign. The model works on automatic feature extraction, making extracting noticeable aspects from the images easier. Some pre-processing techniques such as automatic cropping, radiopaque artifact removal, pectoral muscle removal, and data augmentation are used. Thorough hyperparameter tuning is performed where different learning rates are tested on Adam optimizers. Testing of different optimizers such as Stochastic Gradient Descent (SGD) and RMSProp with the optimal learning rate is also performed. The model uses both max and average pooling to determine the best outcome. Further, SHAP (Shapley Additive Explanations) is incorporated to enhance explainability and provide deeper insights into the decision-making process. The model is tested on standard datasets, including DDSM, MIAS, and INbreast for mammograms, and BUS and MBU for ultrasounds. It achieves high classification accuracies of 97.45%, 96.30%, 98.80%, 99.25%, and 99.97% on these datasets, respectively. The proposed model has undergone evaluation using various performance metrics to assess its effectiveness in analyzing the multimodal dataset.</p>