Dr-SAM: U-Shape Structure Segment Anything Model for Generalizable Medical Image Segmentation

Medical image segmentation plays a pivotal role in computer-assisted medical diagnosis, contributing to precise diagnostics, treatment strategizing, and disease tracking. However, the availability of annotated data for medical image segmentation remains restricted, and conventional approaches predominantly rely on bespoke models with limited adaptability across diverse tasks. In this research, we introduce DrSAM, a foundation model for universal medical image segmentation. This model possesses two crucial attributes: (i) the retention and utilization of pre-trained SAM model weights, while introducing a minimal number of supplementary parameters and computations; (ii) the incorporation of a trainable U-shaped residual network and a Medical Output Token designed to capture the distinctive features at various levels within medical images and enhance mask granularity. DrSAM, a fine-tuned model with a subset of medical image datasets, surpasses existing state-of-the-art segmentation foundation models. DrSAM holds substantial potentials for automating medical image segmentation. Code is available at https://github.com/huoxiangzuo/Doctor-SAM .