Multi-modal Deep Learning with Spatial Transformers for Biparametric MRI Prostate Cancer Classification
摘要
Prostate cancer (PCa) is a leading cause of cancer-related morbidity in men globally, requiring early and accurate diagnosis for effective treatment. Biparametric magnetic resonance imaging (bpMRI) offers a non-invasive approach to PCa detection, potentially reducing the need for more invasive procedures. This paper introduces a novel multi-modal deep learning model for PCa classification using bpMRI, achieving state-of-the-art performance. Our architecture integrates spatial transformer networks (STNs) for spatial alignment of T2-weighted and diffusion-weighted images, convolutional encoders for robust feature extraction, and attention mechanisms to focus on diagnostically important regions. Evaluated on over 15,000 bpMRI slices from the PI-CAI challenge using 5-fold cross-validation, the model demonstrates excellent performance, achieving an overall accuracy of 97% ± 0.2%, a balanced accuracy of 93% ± 0.8%, an AUC of 0.93 ± 0.015, and an MCC of 0.84 ± 0.011. Notably, the model generalises effectively across all International Society of Urological Pathology (ISUP) grades, indicating its potential for robust risk stratification. Ablation studies confirm the synergistic contribution of the STN and attention mechanisms to the model’s performance. Analysis of misclassified slices reveals a tendency for errors to occur near tumor boundaries or with very small lesions (often <1% of the slice area), suggesting potential refinements to labeling practices and highlighting areas for future model development.