<p>Medical image anomaly detection (AD) is crucial for early disease diagnosis, yet it faces challenges such as data heterogeneity and scarcity of annotated samples. This paper introduces a text-adapted few-shot training framework using CLIP, which extends the text encoder to incorporate fine-grained descriptions and introduces a text feature adapter for better alignment with image representations. A text-image feature alignment module and a contrastive learning mechanism are presented to enhance cross-modal integration and the distinction between normal and abnormal samples. Experimental evaluations on six medical imaging datasets demonstrate that our method significantly outperforms state-of-the-art techniques in both classification and segmentation tasks, achieving an average improvement of 1.13% in AUC. The implementation code is available at <a href="https://github.com/clownddd/TAFT">https://github.com/clownddd/TAFT</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing medical anomaly detection via text-adapted few-shot learning with visual-language models

  • Keming Mao,
  • Shengbin Hou,
  • Haoming Fang,
  • Jianzhe Zhao,
  • Xinlu Xiao

摘要

Medical image anomaly detection (AD) is crucial for early disease diagnosis, yet it faces challenges such as data heterogeneity and scarcity of annotated samples. This paper introduces a text-adapted few-shot training framework using CLIP, which extends the text encoder to incorporate fine-grained descriptions and introduces a text feature adapter for better alignment with image representations. A text-image feature alignment module and a contrastive learning mechanism are presented to enhance cross-modal integration and the distinction between normal and abnormal samples. Experimental evaluations on six medical imaging datasets demonstrate that our method significantly outperforms state-of-the-art techniques in both classification and segmentation tasks, achieving an average improvement of 1.13% in AUC. The implementation code is available at https://github.com/clownddd/TAFT.