Scholarlink

<p>Training medical vision-language models (VLMs) typically demands millions of image-text pairs to achieve versatility and reasoning, posing significant challenges in data acquisition. We propose ConceptVLM, a novel data-efficient fine-tuning paradigm that transforms general-domain VLMs into specialized medical ones with minimal labeled data, integrating medical knowledge without disrupting the model’s existing general capabilities. Central to our approach is a key concept-aware training strategy, building a structured medical concept dictionary and employing masked attention to guide the model’s focus toward essential clinical concepts. This focused fine-tuning enhances domain-specific comprehension while preserving the model’s reasoning abilities and response diversity. Experiments across multimodal medical benchmarks show ConceptVLM achieves state-of-the-art results using only 1% of the original training data, outperforming traditional methods reliant on large-scale QA datasets. These findings challenge the prevailing reliance on extensive annotated corpora, demonstrating key concept-guided tuning as a viable path to developing cognitively capable medical VLMs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Key concept learning for medical vision language model with reasoning capabilities

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月30日

查看全文
  • Wei Lou,
  • Yue Wu,
  • Pusheng Xu,
  • Weiyi Zhang,
  • Xiaolan Chen,
  • Jiancheng Yang,
  • Mingguang He,
  • Danli Shi

摘要

Training medical vision-language models (VLMs) typically demands millions of image-text pairs to achieve versatility and reasoning, posing significant challenges in data acquisition. We propose ConceptVLM, a novel data-efficient fine-tuning paradigm that transforms general-domain VLMs into specialized medical ones with minimal labeled data, integrating medical knowledge without disrupting the model’s existing general capabilities. Central to our approach is a key concept-aware training strategy, building a structured medical concept dictionary and employing masked attention to guide the model’s focus toward essential clinical concepts. This focused fine-tuning enhances domain-specific comprehension while preserving the model’s reasoning abilities and response diversity. Experiments across multimodal medical benchmarks show ConceptVLM achieves state-of-the-art results using only 1% of the original training data, outperforming traditional methods reliant on large-scale QA datasets. These findings challenge the prevailing reliance on extensive annotated corpora, demonstrating key concept-guided tuning as a viable path to developing cognitively capable medical VLMs.