Stroke is a leading cause of death and disability worldwide, affecting 12 million people each year. Atrial fibrillation (AF), the most common cardiac arrhythmia, increases risk of stroke 5-fold. Current stratification strategies rely on comorbidity-based risk scores, such as CHA2DS2–VASc, to select high-risk patients suitable for anticoagulation. However, such approaches are highly empirical and have significant limitations, warranting improved stratification strategies. Integrating patient medical imaging data has been proven effective in other cardiology domains and may enhance stroke risk assessment. This study uses explainable convolutional neural network (CNN) and random forest models to predict stroke incidence in high-risk patients from coronary CT angiography data and electronic health records, and to identify early biomarkers of stroke. The models were validated on unseen data using cross validation and compared to CHA2DS2–VASc’s predictions. Shapley additive explanations (SHAP) and Grad-CAM were used to identify key risk factors. The random forest model and the best performing CNN achieved testing AUC of 0.74, 95% CI [0.65–0.82] and 0.81, 95% CI [0.70–0.89] respectively, outperforming CHA2DS2–VASc (0.54, 95% CI [0.45–0.62]). The explainability methods showed the left atrium and left atrial appendage were the most discriminative anatomical features, while BMI and age were some of the most important risk factors of AF-related stroke. In conclusion, this study highlights the power of machine learning and medical imaging in stroke incidence prediction and identifies important structural and clinical biomarkers for patient stratification.