RetBench: Which Ophthalmic Foundation Model Performs Best and Why?
摘要
Foundation models (FMs) have shown great promise in medical image analysis by improving generalization across diverse downstream tasks. In ophthalmology, several FMs have recently emerged, but there is still no clear answer to a fundamental question: Which FM performs the best? Are they equally good across different tasks? The lack of standardized benchmarks makes it difficult to fairly compare these models. To address this gap, we propose RetBench, a comprehensive evaluation suite covering both ophthalmic disease detection (glaucoma, diabetic retinopathy, and age-related macular degeneration) and systemic disease prediction (diabetes and hypertension) based on retinal imaging. We benchmarked four state-of-the-art FMs (RETFound, VisionFM, RetiZero, and DINORET) using standardized datasets from multiple countries and evaluated their performance comprehensive metrics. Our results show that DINORET, RetiZero, and RETFound perform similarly across most tasks. We recommend using DINORET as the base model for glaucoma detection. For instance, RetiZero achieved an average AUC of 0.92 on external validation datasets for systemic disease prediction, compared to 0.88 by the next best model, and showed particularly strong generalization in diabetes prediction. These findings provide an evidence-based answer to the above questions and highlight future directions for improving the clinical applicability of ophthalmic foundation models.