Breast cancer histopathology image classification using an ensemble of optimized pretrained models with a trainable ensemble strategy classifier

Breast cancer stands among the crucial health problems and is deemed the second foremost reason of cancer in the world. Premature detection of malignancy in breast can effectively help patients receive successful treatment and grant them great chance of survival. Nevertheless, the microscopic histopathological examination of breast cancer is strenuous and challenging and may result in a considerable degree of inconsistency between pathologists. For this reason, an automated diagnostic model can help them enhance the diagnostic operation and reduce the manual workload analysis. However, existing deep learning approaches tailored for breast cancer histopathology image classification maintain a full reliance on single pre-trained models in the final classification stage, and the training of their models lacks a clear strategy for the selection of the model hyperparameters. In addition, they have paid less attention in optimizing important hyperparameters like the number of layer to freeze when fine-tuning their models. These limitations work against attaining the optimal performance of their models and the accuracy of classification.

In this paper, we propose a new ensemble architecture to improve both the binary and sub-class classification performances of breast cancer histopathological images using three efficient pre-trained deep learning models (Resnet50, Resnet101, and InceptionV3), together with the use of Bayesian optimization algorithm and an enhanced ensemble strategy. Initially, transfer learning strategy is utilized to reuse the pre-trained weights of the three deep learning models from ImageNet recognition experiment as initial weights. Next, Bayesian optimization algorithm with fivefold cross-validation is assigned for individually retraining each contributing model to unlock the optimal capacity of each model by locating the optimum hyperparameters, including the optimum number of frozen layers for each model. Then, a proposed ensemble strategy is followed by employing the features of the last pool layer of each contributing model for both training and validation data to train an optimized stack of fully connected layers and activation functions to find the best mapping between the input and outputs.

The proposed ensemble with Bayesian optimization and the ensemble strategy was applied on the publicly available BreakHis dataset and attained accuracies up to 99.5% and 97.16% for the binary and multi-class classification tasks, respectively.

The proposed ensemble strategy performed better than each contributing model alone and the other ensemble strategy techniques such as hard and soft voting. Additionally, the proposed ensemble outperforms the recent state-of-the-art approaches for binary and multi-class classification of breast cancer histopathology images.