Generating Chest Radiology Report Findings Using a Multimodal Method

Automatic report generation from chest x-ray imaging (CXR) could potentially alleviate the workload of radiologists and improve clinical efficacy. We introduce a multimodal approach, that integrates radiology images with text describing patients’ indications, to generate the findings section in radiology reports. We instantiate this approach by building on two existing methods, R2Gen and CvT2DistilGPT2. We report experiments on two public datasets, MIMIC-CXR and IU X-ray, using evaluation metrics for natural language generation and clinical efficacy assessment. Results show that improvements across all metrics are obtained through the incorporation of indications text. For example, we obtain 35% and 8% increases in BLEU-4 and F1 scores, respectively.