3D Vision–Language Models with Segmentation-Guided Multimodal Data for Spinal MRI Report Generation
摘要
Automated radiology report generation using vision–language models (VLMs) holds significant promise for improving clinical workflow and diagnostic consistency. However, most existing approaches are limited to 2D image inputs and lack explicit incorporation of anatomical priors. In this study, we present a 3D-aware VLM framework designed to generate radiology reports from volumetric spine MRI scans while integrating anatomical segmentation masks to guide clinical relevance. We evaluate four input configurations: a baseline model using unsegmented MRI volumes, and three segmentation-aware variants—V1 (T1-weighted + segmentation), V2 (T2-weighted + segmentation), and V3 (T1- and T2-weighted + segmentation). Quantitative results across five evaluation metrics (BLEU, ROUGE-1, ROUGE-L, METEOR, and BERTScore) show that all segmentation-based variants significantly outperform the baseline, with V3 achieving the highest lexical accuracy and overall report quality. These findings underscore the value of spatial priors and multimodal fusion in improving the generation of structured, clinically meaningful spinal MRI reports.