A Novel Approach for Brain Tumor Segmentation with SEEM

Brain tumor segmentation is an aspect of analyzing images and making diagnoses, which greatly assists in planning treatment and monitoring progress. The field has seen advancements in deep learning methods with the use of convolutional neural networks (CNNs) leading to a notable shift, in how brain tumor segmentation is approached. Segmenting brain tumors accurately is a critical task in medical imaging that can significantly impact diagnosis and treatment planning. In this study, we explore the potential of zero-shot learning methodologies utilizing large language models (LLMs) named “Segment Everything Everywhere All At Once (SEEM)” for brain tumor segmentation using the brain tumor segmentation (BraTS) dataset which is used and openly accessible datasets among researchers. The zero-shot approach leverages the pre-trained knowledge of LLMs to perform segmentation without the need for extensive task-specific training, offering a novel solution for medical image analysis. We benchmarked the performance of this zero-shot segmentation technique against the Segment Anything Model (SAM), a state-of-the-art method in image segmentation. The evaluation metrics used to compare the performance of these approaches include the Dice coefficient, Intersection over Union (IoU), F1 score, and accuracy. Our findings demonstrate that while the LLM-based zero-shot approach is promising, particularly in scenarios with limited labeled data, SAM generally outperforms it in terms of segmentation accuracy and consistency. This research highlights the potential and limitations of using LLMs in medical image segmentation tasks and suggests pathways for future improvement in zero-shot learning methodologies for clinical applications.