Efficient Foundation Model Pre-training on Mixed Retina Images from Similar Modalities
摘要
Foundation models have demonstrated transformative potential across diverse domains, yet their development often requires extensive datasets and computational resources, necessitating efficient pre-training strategies. In this work, we hypothesise that introducing medical images from similar modalities can serve as an efficient way for data augmentation, potentially alleviating the stress of collecting substantial data and computational resources. We present RETFusion, a retinal foundation model pre-training on mixed retinal images from similar modalities, and evaluate it on extensive experiments. Specifically, we mix fundus fluorescein angiography (FFA) images, a modality with high contrast for lesion detection but rarely collected, with color fundus photography (CFP), commonly collected but with low contrast, for foundation model pre-training. The model is evaluated on clinically relevant applications with both FFA and CFP images. The results demonstrate that, although pre-trained on light data and computational resources, RETFusion achieved competitive performance in the CFP task compared to the state-of-the-art model RETFound and performed best in FFA tasks. Our findings suggest that pre-training on mixed data from similar modalities offers a practical and efficient solution for foundation model development in resource-constrained scenarios. The RETFusion code is available at: https://github.com/a-yayaya/RETFusion .