Automated radiology report generation has the potential to assist radiologists and improve diagnostic accuracy. While prior works incorporate medical knowledge graphs to mitigate data bias, their limited scope and labor-intensive construction hinder scalability across domains. To overcome these limitations, we propose a novel framework that automatically constructs and adapts generalizable clinical knowledge from diverse datasets for radiology report generation. Unlike static knowledge graphs, our method simulates human cognitive processes by extracting disease-related keywords from medical reports, forming a unified and transferable knowledge representation without requiring extra medical annotations. To seamlessly integrate knowledge with visual features, we propose a multi-grained alignment strategy that enhances contextual understanding, enabling the model to leverage knowledge to progressively focus on diverse abnormalities. This strategy consists of two key components: (1) a coarse-grained branch that employs standard self-attention to establish global vision-knowledge associations, and (2) a Knowledge-Visual Bidirectional Alignment (KVBA) module, which refines fine-grained vision-knowledge representations through bidirectional cross-attention. Evaluated on public IU-Xray and MIMIC-CXR benchmarks, our method successfully enhances the accuracy of disease-related keywords mentioned in medical reports and achieves competitive results in clinical efficacy metrics in report generation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Towards Generalizable Clinical Knowledge Discovery for Radiology Report Generation

  • Xiaomin Chen,
  • Qi Chen,
  • Minh Hieu Phan,
  • Qi Wu,
  • Jian Chen,
  • Yutong Xie

摘要

Automated radiology report generation has the potential to assist radiologists and improve diagnostic accuracy. While prior works incorporate medical knowledge graphs to mitigate data bias, their limited scope and labor-intensive construction hinder scalability across domains. To overcome these limitations, we propose a novel framework that automatically constructs and adapts generalizable clinical knowledge from diverse datasets for radiology report generation. Unlike static knowledge graphs, our method simulates human cognitive processes by extracting disease-related keywords from medical reports, forming a unified and transferable knowledge representation without requiring extra medical annotations. To seamlessly integrate knowledge with visual features, we propose a multi-grained alignment strategy that enhances contextual understanding, enabling the model to leverage knowledge to progressively focus on diverse abnormalities. This strategy consists of two key components: (1) a coarse-grained branch that employs standard self-attention to establish global vision-knowledge associations, and (2) a Knowledge-Visual Bidirectional Alignment (KVBA) module, which refines fine-grained vision-knowledge representations through bidirectional cross-attention. Evaluated on public IU-Xray and MIMIC-CXR benchmarks, our method successfully enhances the accuracy of disease-related keywords mentioned in medical reports and achieves competitive results in clinical efficacy metrics in report generation.