Automated radiology report generation using vision–language models (VLMs) holds significant promise for improving clinical workflow and diagnostic consistency. However, most existing approaches are limited to 2D image inputs and lack explicit incorporation of anatomical priors. In this study, we present a 3D-aware VLM framework designed to generate radiology reports from volumetric spine MRI scans while integrating anatomical segmentation masks to guide clinical relevance. We evaluate four input configurations: a baseline model using unsegmented MRI volumes, and three segmentation-aware variants—V1 (T1-weighted + segmentation), V2 (T2-weighted + segmentation), and V3 (T1- and T2-weighted + segmentation). Quantitative results across five evaluation metrics (BLEU, ROUGE-1, ROUGE-L, METEOR, and BERTScore) show that all segmentation-based variants significantly outperform the baseline, with V3 achieving the highest lexical accuracy and overall report quality. These findings underscore the value of spatial priors and multimodal fusion in improving the generation of structured, clinically meaningful spinal MRI reports.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

3D Vision–Language Models with Segmentation-Guided Multimodal Data for Spinal MRI Report Generation

  • Hoda Helmy,
  • Abdullah Hosseini,
  • Ahmed Ibrahim,
  • Asfand Baig-Mirza,
  • Ahmed-Ramadan Sadek,
  • Ahmed Serag

摘要

Automated radiology report generation using vision–language models (VLMs) holds significant promise for improving clinical workflow and diagnostic consistency. However, most existing approaches are limited to 2D image inputs and lack explicit incorporation of anatomical priors. In this study, we present a 3D-aware VLM framework designed to generate radiology reports from volumetric spine MRI scans while integrating anatomical segmentation masks to guide clinical relevance. We evaluate four input configurations: a baseline model using unsegmented MRI volumes, and three segmentation-aware variants—V1 (T1-weighted + segmentation), V2 (T2-weighted + segmentation), and V3 (T1- and T2-weighted + segmentation). Quantitative results across five evaluation metrics (BLEU, ROUGE-1, ROUGE-L, METEOR, and BERTScore) show that all segmentation-based variants significantly outperform the baseline, with V3 achieving the highest lexical accuracy and overall report quality. These findings underscore the value of spatial priors and multimodal fusion in improving the generation of structured, clinically meaningful spinal MRI reports.