MedSeg3D: Leveraging Implicit Neural Representation and 2D Foundation Models for High-Fidelity 3D Medical Image Segmentation
摘要
High-quality 3D medical image segmentation is crucial for clinical diagnosis but is constrained by the need for annotated volumetric data and the computational demands of 3D models. We present MedSeg3D, a novel framework that performs high-fidelity 3D segmentation directly from multiview 2D images without requiring 3D ground-truth annotations. MedSeg3D leverages the 2D foundation model UniverSeg to generate coarse segmentation masks, which are refined using an Implicit Neural Representation (INR) and differentiable volume rendering. At its core, a Light-MLP maps spatial coordinates to segmentation probabilities, enabling efficient training and fine anatomical detail. We further introduce a Soft Binary Mapping (SBM) function with three integration strategies to improve voxel-level sharpness and boundary accuracy. Experiments on chest and knee datasets demonstrate strong segmentation performance, cross-view consistency, and data efficiency. MedSeg3D establishes a scalable approach to 3D segmentation by effectively bridging 2D foundation models with implicit volumetric learning.