Scholarlink

Automated medical report generation lacks precise correspondences between anatomical regions and their descriptive sentences, limiting clinical accuracy and interpretability. We propose REALM, a data-efficient framework that achieves fine-grained alignment between image regions and report sentences. Our approach uses large language models to automatically annotate sentences with anatomical regions, while a reinforcement learning-based module localizes corresponding visual regions with minimal supervision. Through contrastive learning, we establish explicit alignments between visual regions and textual descriptions. Experiments on IU X-ray and MIMIC-CXR datasets demonstrate superior performance in both language generation metrics (BLEU-4: 0.201 vs 0.181) and clinical accuracy (F1: 0.384 vs 0.373), while reducing dependency on large-scale manual annotations.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

REALM: Reinforcement Learning-Based Cross-Modal Alignment for Fine-Grained Medical Report Generation

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Pengrui Teng,
  • Wenjian Liu,
  • Yueyue Wang,
  • Hongyu Yang,
  • Meng Xing

摘要

Automated medical report generation lacks precise correspondences between anatomical regions and their descriptive sentences, limiting clinical accuracy and interpretability. We propose REALM, a data-efficient framework that achieves fine-grained alignment between image regions and report sentences. Our approach uses large language models to automatically annotate sentences with anatomical regions, while a reinforcement learning-based module localizes corresponding visual regions with minimal supervision. Through contrastive learning, we establish explicit alignments between visual regions and textual descriptions. Experiments on IU X-ray and MIMIC-CXR datasets demonstrate superior performance in both language generation metrics (BLEU-4: 0.201 vs 0.181) and clinical accuracy (F1: 0.384 vs 0.373), while reducing dependency on large-scale manual annotations.