Scholarlink

Fundus images and optical coherence tomography (OCT) provide complementary diagnostic information from retinal surface structures and deep tissue tomograms, respectively, playing a critical role in early screening of blinding diseases such as diabetic retinopathy and glaucoma. However, the differences in imaging principles between the two modalities (e.g., color texture of fundus images vs. tomographic grayscale features of OCT) lead to significant heterogeneity in feature spaces. Traditional single-modal models (e.g., ResNet-based single-image classifiers) often suffer from incomplete feature representation in complex lesion recognition due to the lack of multimodal information interaction. To address this, this paper proposes an early-stage feature fusion framework based on dual MobileNetV3 networks, which extracts discriminative features through modality-specific networks and enhances pathological region focusing using a self-attention mechanism. The framework achieved a 98.9% test accuracy on public datasets, presenting a novel optimized approach for multimodal medical image diagnosis.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal Deep Learning for Retinal Disease Diagnosis

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Dongmei Wang,
  • Yuansong Cai,
  • Wanli Qiao,
  • Menglei Liu

摘要

Fundus images and optical coherence tomography (OCT) provide complementary diagnostic information from retinal surface structures and deep tissue tomograms, respectively, playing a critical role in early screening of blinding diseases such as diabetic retinopathy and glaucoma. However, the differences in imaging principles between the two modalities (e.g., color texture of fundus images vs. tomographic grayscale features of OCT) lead to significant heterogeneity in feature spaces. Traditional single-modal models (e.g., ResNet-based single-image classifiers) often suffer from incomplete feature representation in complex lesion recognition due to the lack of multimodal information interaction. To address this, this paper proposes an early-stage feature fusion framework based on dual MobileNetV3 networks, which extracts discriminative features through modality-specific networks and enhances pathological region focusing using a self-attention mechanism. The framework achieved a 98.9% test accuracy on public datasets, presenting a novel optimized approach for multimodal medical image diagnosis.