Towards Generalizable Clinical Knowledge Discovery for Radiology Report Generation
摘要
Automated radiology report generation has the potential to assist radiologists and improve diagnostic accuracy. While prior works incorporate medical knowledge graphs to mitigate data bias, their limited scope and labor-intensive construction hinder scalability across domains. To overcome these limitations, we propose a novel framework that automatically constructs and adapts generalizable clinical knowledge from diverse datasets for radiology report generation. Unlike static knowledge graphs, our method simulates human cognitive processes by extracting disease-related keywords from medical reports, forming a unified and transferable knowledge representation without requiring extra medical annotations. To seamlessly integrate knowledge with visual features, we propose a multi-grained alignment strategy that enhances contextual understanding, enabling the model to leverage knowledge to progressively focus on diverse abnormalities. This strategy consists of two key components: (1) a coarse-grained branch that employs standard self-attention to establish global vision-knowledge associations, and (2) a Knowledge-Visual Bidirectional Alignment (KVBA) module, which refines fine-grained vision-knowledge representations through bidirectional cross-attention. Evaluated on public IU-Xray and MIMIC-CXR benchmarks, our method successfully enhances the accuracy of disease-related keywords mentioned in medical reports and achieves competitive results in clinical efficacy metrics in report generation.