Multi-modal clear cell renal cell carcinoma grading with the segment anything model

Clear cell renal cell carcinoma (ccRCC) is a prevalent kidney disease, accounting for more than 75% of renal cell carcinoma (RCC) and approximately 3.8% of human malignancies. Early grading of ccRCC is crucial in guiding personalized treatment plans, and it is of great significance in clinical decision-making and prognosis evaluation, but ccRCC grade is usually obtained by postoperative pathology. The purpose of this study is to develop an automated non-invasive approach based on deep learning methods for accurate segmentation and preoperative grading of ccRCC tumors into low-grade and high-grade categories. CT images, radiomics features, and patient data are enrolled for our ccRCC grading framework. Firstly, we train a model for kidney tumor segmentation on CT images based on the recently released Segment Anything Model (SAM) with simple yet effective strategies. Secondly, we utilize the pretrained image encoder from the segmentation model to propose a contrastive learning-based approach, in order to learn the mutual information between the images and radiomics. Lastly, we train machine learning-based models for ccRCC grading, using image features, radiomics features and patient data. Our proposed segmentation method has demonstrated superior performance in kidney tumor segmentation, outperforming the original SAM with an average improvement of 6.03% in Dice Similarity Coefficient (DSC) and 16.24% in surface Dice Similarity Coefficient (sDSC). While for ccRCC grading, our method achieves an accuracy of 82.50%, superior to other comparative methods. The experimental results demonstrate the efficacy of our methods in accurately segmenting ccRCC tumors and providing reliable preoperative grading information. These findings highlight the significant clinical potential of our approach.