Foundation models (FMs) have shown great promise in medical image analysis by improving generalization across diverse downstream tasks. In ophthalmology, several FMs have recently emerged, but there is still no clear answer to a fundamental question: Which FM performs the best? Are they equally good across different tasks? The lack of standardized benchmarks makes it difficult to fairly compare these models. To address this gap, we propose RetBench, a comprehensive evaluation suite covering both ophthalmic disease detection (glaucoma, diabetic retinopathy, and age-related macular degeneration) and systemic disease prediction (diabetes and hypertension) based on retinal imaging. We benchmarked four state-of-the-art FMs (RETFound, VisionFM, RetiZero, and DINORET) using standardized datasets from multiple countries and evaluated their performance comprehensive metrics. Our results show that DINORET, RetiZero, and RETFound perform similarly across most tasks. We recommend using DINORET as the base model for glaucoma detection. For instance, RetiZero achieved an average AUC of 0.92 on external validation datasets for systemic disease prediction, compared to 0.88 by the next best model, and showed particularly strong generalization in diabetes prediction. These findings provide an evidence-based answer to the above questions and highlight future directions for improving the clinical applicability of ophthalmic foundation models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

RetBench: Which Ophthalmic Foundation Model Performs Best and Why?

  • Ke Zou,
  • Jocelyn Hui Lin Goh,
  • Yukun Zhou,
  • Samantha Min Er Yew,
  • Meng Wang,
  • Huazhu Fu,
  • Ching-Yu Cheng,
  • Yih Chung Tham

摘要

Foundation models (FMs) have shown great promise in medical image analysis by improving generalization across diverse downstream tasks. In ophthalmology, several FMs have recently emerged, but there is still no clear answer to a fundamental question: Which FM performs the best? Are they equally good across different tasks? The lack of standardized benchmarks makes it difficult to fairly compare these models. To address this gap, we propose RetBench, a comprehensive evaluation suite covering both ophthalmic disease detection (glaucoma, diabetic retinopathy, and age-related macular degeneration) and systemic disease prediction (diabetes and hypertension) based on retinal imaging. We benchmarked four state-of-the-art FMs (RETFound, VisionFM, RetiZero, and DINORET) using standardized datasets from multiple countries and evaluated their performance comprehensive metrics. Our results show that DINORET, RetiZero, and RETFound perform similarly across most tasks. We recommend using DINORET as the base model for glaucoma detection. For instance, RetiZero achieved an average AUC of 0.92 on external validation datasets for systemic disease prediction, compared to 0.88 by the next best model, and showed particularly strong generalization in diabetes prediction. These findings provide an evidence-based answer to the above questions and highlight future directions for improving the clinical applicability of ophthalmic foundation models.