MedCapNet: A Novel Approach to Medical Image Captioning
摘要
Medical image captioning is crucial for automating the generation of accurate textual descriptions for medical images. This paper introduces MedCapNet, a novel encoder-decoder architecture designed to bridge the gap between visual and textual modalities in medical imaging. The model incorporates a Swin Transformer and Enhancement Encoder, allowing for the efficient extraction and refinement of both patch-level and global-level features from medical images. A Transformer block with a Fusion Module is utilized by the decoder to seamlessly integrate visual and linguistic information. A key innovation is Dual-Scale Masked Multi-Head Self-Attention, which enhances the model’s ability to effectively capture long-range dependencies and fine-grained details. Our model was evaluated on ROCO v2, achieving state-of-the-art performance with scores of 0.647, 0.239, and 0.094 for BERTScore, CIDEr, and METEOR, respectively.