TransDSUNet: a novel transformer-based deep supervision UNet with attention-guided denoising for brain tumor segmentation

Brain tumors represent one of the most disease that impact heavily the life expectancy leading to the death in general cases. Consequently, early diagnosis and efficient treatment are crucial to extend patient lifespan, improve prognosis and achieve better outcomes. MRI is a widely used imaging technique to confirm the diagnosis; however, manual tumor segmentation is time-consuming and susceptible to errors. Consequently, AI methods have become important to automate and accelerate this task. Thus, DL based methods still face challenges related to imaging artifacts, imbalance data and computational complexity. To overcome these challenges, we introduce a novel transformer UNet with deep supervision and attention-guided denoising model dubbed as TransDSUNet. The proposed architecture uses in the preprocessing stage the attention-guided denoising autoencoder (AGDAE) with squeeze–excitation module (DS) to denoise and increase the resolution of multimodal MRI input. Furthermore, in the segmentation stage, we incorporated the transformer into the bottleneck part and deep supervision into the decoder which made the model able to efficiently capture relevant features and enhance accuracy of brain tumor segmentation. This pipeline, conducted on the BraTS 2020 and 2021 datasets, demonstrated an optimized performance. The results achieved a mean Dice score of 97.84%, precision 97.34% and IoU score of 96.63% on BraTS 2020 and a mean Dice score of 98.27%, precision 97.72% and IoU score of 96.60% on BraTS 2021. Therefore, the proposed model achieved high results compared to existing state of the art models. Our code is available upon acceptance at: https://github.com/sohe94/TransDSUNet-Brain-Tumor-Segmentation