Adapting Vision Language Models for Structured Clinical Description Generation in Mammography
摘要
Automated report generation for mammographic images remains a significant challenge in clinical radiology, particularly in resource-constrained environments where specialized artificial intelligence (AI) infrastructure is limited. This work evaluates open-access vision-language models (VLMs) and fine-tunes the best-performing one to generate descriptions of mammographic images. We compare four baseline models: BLIP, MiniGPT-4, GIT, and CLIP+GPT-2, using a multi-agent LLM evaluation protocol with GPT-4o mini, GPT-4.1 mini, and Gemini 2.5 Flash. Our experiments on the VinDr-Mammo dataset show that CLIP+GPT-2 achieves the highest normalized clinical relevance score (0.4687). After fine-tuning on 15,000 training images, the model demonstrates improvements: BLEU scores increased by 3282.6%, ROUGE-L by 2055.3%, and METEOR by 1737.2%. These results demonstrate that accessible and computationally efficient vision-language models can be effectively adapted for structured clinical description generation, offering a practical solution for deploying AI in mammography screening programs.