DeepDepth: Prediction of O(6)-methylguanine-DNA methyltransferase genotype in glioblastoma patients using multimodal representation learning based on deep feature fusion

Representation learning aims to extract meaningful features from medical images that are often multimodal, i.e., captured using multiple imaging modalities, to provide a more comprehensive understanding of the underlying anatomy and pathology. The primary objective is to improve the quality of features that can help in accurate diagnosis, disease detection, and treatment planning. With more than one modality, the extracted features must be integrated efficiently to capture the complex relationships between them. Integrating features in medical imaging-based classification problems, such as those involving magnetic resonance imaging-based diagnosis, remains a significant challenge. Current approaches to multimodal medical imaging classification often focus on features from a single modality or concatenate them into a high-dimensional feature vector, resulting in overfitting. Intermediate methods like deep fusion that combine features at a mid-level or deeper level have shown promise in offering a better representation. Depthwise convolution is a promising alternative in the deep learning paradigm that has demonstrated strong performance while requiring less computation. This paper presents a novel method in deep fusion based on depthwise 1D convolution. The proposed architecture is compared with other methods based on different fusion techniques and machine learning classifiers. The effectiveness of this method is demonstrated through experiments involving the brain tumor segmentation (BraTs-21) competition, Task-2, which involves predicting O(6)-methylguanine-DNA methyltransferase promoter status, a biomarker for glioblastoma, as a binary classification problem using multimodal magnetic resonance images. The proposed approach involves calculating radiomic features directly from magnetic resonance images and deep features from pre-trained deep learning models, selecting valuable features from both sets and then deep feature fusion using depthwise 1D convolution followed by fully connected layers for classification. In this approach, the deep fusion and classifier part is trainable. To make the proposed framework generalizable, we have trained and tested the model with the public BraTs-21 dataset and externally tested the model on a public Lumiere dataset. The experimental results show the promising performance of this approach. The proposed method got the best AUC score of 0.748 with minimum BCE loss of 0.62 on BraTs-21 as compared to other methods.