Self-supervised approach for diabetic retinopathy severity detection using vision transformer

Diabetic retinopathy (DR) is a diabetic condition that affects vision, despite the great success of supervised learning and Conventional Neural Networks (CNNs), it’s still challenging to detect the severity of DR at an early stage. The label-intensive nature of supervised learning and the limited scalability of CNNs inhibit exploiting tons of unlabeled medical images that can be useful for capturing rich domain-specific features. The local feature representations from CNNs and their inability to scale well with increased unlabeled data lead to indiscriminative representations not effective for the downstream task. Hence in this work, vision transformer-based paradigm for diabetic retinopathy (DR) severity detection is presented by undertaking a scalable learning approach for model building. Self-supervised learning (SSL) framework with transformer architecture is used to classify fundus images to one of the DR categories by pre-training the model on tons of unlabeled fundus images followed by supervised training on different fractions of labeled data. The performance of the proposed transformer based self-supervised DR detection models (DINO[ViT]DINO[ViT], MAE[ViT]MAE[ViT], MSN[ViT]MSN[ViT]) is analysed at different data regimes. The experimental results showcase that SSL with vision transformer achieves higher performance when pre-trained on large corpus of unlabeled data. One of the proposed DR classifiers (MAE[ViT]) pre-trained using Masked Autoencoder framework on unlabeled fundus images achieves kappa score of 0.9027 when trained on full data regime and achieves a significant performance of 0.8341 on lower data regime (trained on 50 samples of EyePACS training data). The best-performing model MAE[ViT]MAE[ViT] is also inspected for model explainabilityexplainability to infuse trust in the model predictions.