Scholarlink

<p>Early detection of diabetic retinopathy (DR) is crucial for preventing irreversible vision loss; however, existing automated methods often rely on single-modality inputs, thereby limiting the complementary diagnostic value of multimodal imaging. This study presents a cross-attention-based multimodal fusion framework that investigates cross-attention-based representation learning using semantically aligned fundus photography and OCT datasets for the classification of early-stage diabetic retinopathy (E-DR). The proposed architecture utilises EfficientNetB0 and DenseNet121 as modality-specific encoders, followed by a mid-level cross-attention module that dynamically aligns and fuses vascular and structural features. To enable consistent multimodal training, we develop a harmonised label-mapping strategy and a balanced dataset construction pipeline. Extensive experiments across unimodal and fused configurations demonstrate that our fusion model significantly outperforms baseline models, achieving an overall accuracy of 96% and notable improvements in early-stage detection (e.g., F1-score of 83% for Stage 1), comparing to the previous fusion state-of-the-art works that rely on simple concatenation or late-stage integration, our proposed fusion model captures richer cross-modal feature interactions, resulting in improved classification performance on the evaluated datasets. This study provides an exploratory investigation of cross-attention-guided multimodal fusion for retinal image analysis for early DR diagnosis and other ophthalmic screening assessments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Cross-attention-based multimodal fusion for early diabetic retinopathy detection

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月28日

查看全文
  • Jameel Ahmed Bhutto,
  • Ziaur Rahman,
  • Zhonglin He,
  • Fida Hussain Dahri,
  • Asif Ali Laghari

摘要

Early detection of diabetic retinopathy (DR) is crucial for preventing irreversible vision loss; however, existing automated methods often rely on single-modality inputs, thereby limiting the complementary diagnostic value of multimodal imaging. This study presents a cross-attention-based multimodal fusion framework that investigates cross-attention-based representation learning using semantically aligned fundus photography and OCT datasets for the classification of early-stage diabetic retinopathy (E-DR). The proposed architecture utilises EfficientNetB0 and DenseNet121 as modality-specific encoders, followed by a mid-level cross-attention module that dynamically aligns and fuses vascular and structural features. To enable consistent multimodal training, we develop a harmonised label-mapping strategy and a balanced dataset construction pipeline. Extensive experiments across unimodal and fused configurations demonstrate that our fusion model significantly outperforms baseline models, achieving an overall accuracy of 96% and notable improvements in early-stage detection (e.g., F1-score of 83% for Stage 1), comparing to the previous fusion state-of-the-art works that rely on simple concatenation or late-stage integration, our proposed fusion model captures richer cross-modal feature interactions, resulting in improved classification performance on the evaluated datasets. This study provides an exploratory investigation of cross-attention-guided multimodal fusion for retinal image analysis for early DR diagnosis and other ophthalmic screening assessments.