UniSeg-CT: A Foundation Model for Medical Image Segmentation across Cardiac, Oncologic, and Neurovascular Applications

Accurate segmentation of medical images is essential for quantitative analysis, treatment planning, and disease monitoring, yet developing robust models is often hampered by limited annotations and heterogeneous imaging protocols. Self-supervised learning (SSL) offers a promising solution by leveraging large volumes of unlabeled data to learn rich anatomical representations. In this work, we introduce UniSeg-CT, a foundation model for CT-based medical image segmentation, pretrained on a large collection of unlabeled CT scans. We systematically evaluate two state-of-the-art volumetric SSL strategies: Masked Autoencoders (MAE), which learn local and structural details through reconstruction of missing regions, and 3D DINOv2, which captures global anatomical context via self-distillation without labels. Both approaches enable UniSeg-CT to extract rich feature representations without manual annotation, yet they differ in task generalization and structural sensitivity. Fine-tuning was performed on three clinically relevant benchmarks: whole-heart CT segmentation, head and neck tumors and lymph nodes from multimodal PET/CT, and multi-class brain hemorrhage segmentation. Across all tasks, UniSeg-CT pretrained with either MAE or 3D DINOv2 consistently outperformed task-specific baselines. MAE excelled in capturing fine structural details, while 3D DINOv2 provided enhanced global context and volumetric consistency. These results demonstrate the complementary strengths of different SSL strategies and establish UniSeg-CT as a generalizable, scalable foundation model for universal CT-based segmentation, bridging cardiology, oncology, and neurology applications.