Uncertainty-Aware Multimodal Fusion for Reliable Fundus Disease Classification Using a Vision-Language Foundation Model
摘要
Accurate and trustworthy diagnosis of fundus diseases is essential for the safe clinical deployment of artificial intelligence (AI) systems. In this study, we present a novel multimodal diagnostic framework that enhances both classification performance and uncertainty quantification. Our method leverages a pre-trained vision-language foundation model to extract visual features from fundus images and semantic embeddings from disease label texts. Two predictive pathways, supervised classification and image–text similarity, are independently modeled, and their outputs are transformed into belief masses and uncertainty scores using Dirichlet-based evidential modeling. These modality-specific predictions are then integrated using a Dempster–Shafer-inspired uncertainty-aware fusion strategy, which dynamically calibrates trust based on the reliability of each modality. Experimental results on a multi-class fundus disease dataset and two out-of-distribution (OOD) benchmarks show that our method outperforms state-of-the-art baselines in fundus disease diagnosis, achieves over 80% accuracy in OOD detection, and supports uncertainty-guided triaging. These results demonstrate the effectiveness and reliability of our framework for real-world AI-assisted ophthalmic screening.