Zero-shot skin lesion segmentation allows for efficient processing times, without reliance on laborious data gathering for training supervised segmentation models. However, while foundation models such as Segment Anything (SAM) have shown strong ability to segment skin lesions without prior domain-knowledge, they require manual prompt generation. This paper introduces an automated prompt generation method for Medical Segment Anything Model (MedSAM) that uses class activation maps as weak localizations to produce the required bounding box and coordinate point prompts needed for zero-shot segmentation. This classification-to-segmentation method reduces the time required for clinicians to assess a skin lesion, while maintaining a human-in-the-loop approach by providing an initial assessment mask which can be analyzed and refined. Zero-shot and finetuned classification-to-segmentation performance with an array of established CAM-based explanation methods is assessed for both CNN and vision transformer models from ISIC 2017. With examples such as MobileNet-v2 demonstrating favourable zero-shot generalization, actionable segmentation results are produced. Code available at: https://github.com/xraikeele/Classification-to-Segmentation .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Classification-to-Segmentation: Class Activation Mapping for Zero-Shot Skin Lesion Segmentation

  • Matthew J. Cockayne,
  • Marco Ortolani,
  • Baidaa Al-Bander

摘要

Zero-shot skin lesion segmentation allows for efficient processing times, without reliance on laborious data gathering for training supervised segmentation models. However, while foundation models such as Segment Anything (SAM) have shown strong ability to segment skin lesions without prior domain-knowledge, they require manual prompt generation. This paper introduces an automated prompt generation method for Medical Segment Anything Model (MedSAM) that uses class activation maps as weak localizations to produce the required bounding box and coordinate point prompts needed for zero-shot segmentation. This classification-to-segmentation method reduces the time required for clinicians to assess a skin lesion, while maintaining a human-in-the-loop approach by providing an initial assessment mask which can be analyzed and refined. Zero-shot and finetuned classification-to-segmentation performance with an array of established CAM-based explanation methods is assessed for both CNN and vision transformer models from ISIC 2017. With examples such as MobileNet-v2 demonstrating favourable zero-shot generalization, actionable segmentation results are produced. Code available at: https://github.com/xraikeele/Classification-to-Segmentation .