MMHBC-Net: a multi-modal hybrid approach for breast cancer classification
摘要
Early detection of breast cancer is crucial for improving survival rates. Many computer-aided diagnosis (CAD) models have been proposed in the past for the detection of breast cancer. However, analyzing breast cancer across multiple imaging modalities remains a challenge. Different imaging modalities provide complimentary information on the lesion and help in detecting the abnormalities at an early stage. Our study introduces a MMHBC-Net model that utilizes mammogram, ultrasound, and histopathology images to enhance detection accuracy. Pre-processing and augmentation of the images are performed for better accuracy. For extracting features from mammogram and ultrasound images, a deep learning model that includes four convolutional layers followed by a dense layer. For histopathology, handcrafted techniques based on texture and color feature extraction are employed. Data-level fusion is performed among the image modalities and passed through various classifiers, including RF, BAG, DT, ADA, NB, LGBM, XGB, GBC, kNN SVM, and GCN. Based on the classifier’s performance, the top five classifiers for each modality are selected to construct a stacking ensemble classifier for the final classification. Experimentation involves datasets such as DDSM, MIAS, INbreast for mammograms, BUS and MBU for ultrasounds, and BreakHis for histopathology images. The results reveal that the multi-modal CAD system outperforms uni-modal CAD systems by obtaining test accuracy of 99.96, 88.16, 95.33, and 91.79% for combined, mammogram, ultrasound, and histopathology images. This highlights the effectiveness of the multi-modal approach for breast cancer detection.