Integrated Gaussian—GLCM Butterfly Optimization with CNN (IGGBOCNN): A Hybrid Approach for Ovarian Cancer Classification in Medical Image Analysis

Cancer has become a common and often fatal illness. Ovarian cancer is a particularly persistent and aggressive type of gynecologic cancer. Basic epithelial ovarian cancer is classified into four different subcategories: mucinous, serous, clear cell, and endometrioid. The four distinct types in the cytological pictures that the pathologist's eyes and intellect correctly identify are frequently difficult to distinguish from one another. However, there is a great likelihood of inaccuracy if individuals evaluate and determine a large number of photos. Consequently, the investigation has used technological advances in pathology prediction to lower the risk of error and acquire an accurate diagnosis. In this work, we suggest an extensive approach for classifying ovarian cancer that combines conventional ML (machine learning) approaches with optimization strategies. Butterfly Optimization (BO) is used to optimize the model parameters for Support Vector Machines (SVM), Random Forest (RF), and Convolutional Neural Networks (CNN) in particular. Moreover, preprocessing methods are used to improve the quality of input data, such as feature extraction with the Gray Level Co-occurrence Matrix (GLCM) and noise removal with a Gaussian filter. The results of our experiments show that the suggested combination of methods—namely, classification using CNN, the model parameter tuning by BO, the texture feature extraction through GLCM, and the noise removal through Gaussian filter—produces better outcomes in terms of accuracy of 91%, precision of 0.90, and recall of 0.87, respectively. The Python programming language is the analysis device.