A Hybrid UNet and Vision Transformer Architecture with Multi-scale Fusion for Brain Tumor Segmentation
摘要
This research presents an advanced neural network architecture for the segmentation of brain tumors. Our suggested model for brain tumor segmentation incorporates the Vision Transformer (ViT) as a 3D encoder alongside a modified UNet architecture to effectively capture both global and local information in MRI data. The model contains a dual-encoder architecture: a 3D ViT encoder captures global contextual information throughout the MRI volume, while a concurrent convolutional encoder emphasizes localized features. The encoder routes are enhanced by skip connections, facilitating efficient feature transmission between the ViT and UNet layers. In the bottleneck, features from both encoders are integrated via a multi-scale fusion block, augmenting the model's capacity to discern intricate spatial hierarchies and tumor changes. The decoder pathway employs a hierarchical upsampling and multi-scale fusion framework, meticulously rebuilding the final segmentation map with a focus on spatial accuracy. This hybrid model integrates the ViT's capacity to capture long-range relationships with the UNet's proficiency in fine-grained spatial localization, yielding a strong segmentation output on the BraTS 2020 dataset that is adept at addressing the complex and diverse architecture of brain tumors in MRI.