Scholarlink

<p>Medical imaging is crucial for clinical decisions, but manual report writing is time-consuming and error-prone. Automated generation helps professionals avoid errors and save time. Transformer models excel in capturing long-term dependencies, but face challenges in detailed image representation and modeling lengthy descriptions in medical reports. Addressing these challenges, we propose two modules. First, a transformable and context-aware attentive visual extractor (TCAVE) enhances image features by selecting informative spatial-semantic details across scales, improving report quality. TCAVE involves two novel networks in the ResNet 101 architecture, i.e., (i) incorporating a transformable network into ResNet 101’s intermediate layer for spatially invariant features and (ii) integrating a context-aware network into the final ResNet 101 layer for rich multi-scale contextual features. The transformer encoder encodes fine-grained radiology image details using these representations. Second, a selective memory module with relational gating SMMRG is integrated into the transformer decoder to efficiently model long-term dependencies between input-output sequences while maintaining contextual memory. Our model outperforms existing works on the IU X-Ray and MIMIC-CXR datasets.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Semantic-assisted report generation with memory enhanced transformer using context-aware visual extractor

  • 期刊论文

  • 发表日期：

    2025年12月5日

查看全文
  • Divya Peketi,
  • Partha Sarathi Chakraborty,
  • Krishna Mohan Chalavadi,
  • Yen Wei Chen

摘要

Medical imaging is crucial for clinical decisions, but manual report writing is time-consuming and error-prone. Automated generation helps professionals avoid errors and save time. Transformer models excel in capturing long-term dependencies, but face challenges in detailed image representation and modeling lengthy descriptions in medical reports. Addressing these challenges, we propose two modules. First, a transformable and context-aware attentive visual extractor (TCAVE) enhances image features by selecting informative spatial-semantic details across scales, improving report quality. TCAVE involves two novel networks in the ResNet 101 architecture, i.e., (i) incorporating a transformable network into ResNet 101’s intermediate layer for spatially invariant features and (ii) integrating a context-aware network into the final ResNet 101 layer for rich multi-scale contextual features. The transformer encoder encodes fine-grained radiology image details using these representations. Second, a selective memory module with relational gating SMMRG is integrated into the transformer decoder to efficiently model long-term dependencies between input-output sequences while maintaining contextual memory. Our model outperforms existing works on the IU X-Ray and MIMIC-CXR datasets.