APSeg: Auto-prompt Model with Acquired and Injected Knowledge for Nuclear Instance Segmentation and Classification
摘要
Nuclear instance segmentation and classification provide critical quantitative foundations for digital pathology diagnosis. The emergence of the large-scale pretrained foundational model, Segment Anything Model (SAM), has provided a novel solution for nuclear instance segmentation. However, SAM imposes a strong reliance on precise prompts, and its class-agnostic design renders its classification results entirely dependent on the provided prompts. Therefore, we focus on generating prompts with more accurate localization and classification and propose APSeg, Auto-Prompt model with acquired and injected knowledge for nuclear instance Segmentation and classification. APSeg incorporates two knowledge-aware modules: (1) Distribution-Guided Proposal Offset Module (DG-POM), which learns distribution knowledge through density map, and (2) Category Knowledge Semantic Injection Module (CK-SIM), which injects morphological knowledge derived from category descriptions. We conducted extensive experiments on the PanNuke and CoNSeP datasets, demonstrating the effectiveness of our approach. Code is available at https://github.com/hotaru-X/APSeg