Background <p>Early prediction of treatment response to neoadjuvant therapy (NAT) in breast cancer patients can facilitate timely adjustment of treatment regimens. We aimed to develop and validate a MRI-based enhanced self-attention network (MESN) for predicting pathological complete response (pCR) based on longitudinal images at the early stage of NAT.</p> Methods <p>Two imaging datasets were utilized: a subset from the ACRIN 6698 trial (dataset A, <i>n</i> = 227) and a prospective collection from a Chinese hospital (dataset B, <i>n</i> = 245). These datasets were divided into three cohorts: an ACRIN 6698 training cohort (<i>n</i> = 153) from dataset A, an ACRIN 6698 test cohort (<i>n</i> = 74) from dataset A, and an external test cohort (<i>n</i> = 245) from dataset B. The proposed MESN allowed for the integration of multiple timepoint features and extraction of dynamic information from longitudinal MR images before and after early-NAT. We also constructed the Pre model based on pre-NAT MRI features. Clinicopathological characteristics were added to these image-based models to create integrated models (MESN-C and Pre-C), and their performance was evaluated and compared.</p> Results <p>The MESN-C yielded area under the receiver operating characteristic curve (AUC) values of 0.944 (95% CI: 0.906 − 0.973), 0.903 (95%CI: 0.815 − 0.965), and 0.861 (95%CI: 0.811 − 0.906) in the ACRIN 6698 training, ACRIN 6698 test and external test cohorts, respectively, which were significantly higher than those of the clinical model (AUC: 0.720 [95%CI: 0.587 − 0.842], 0.738 [95%CI: 0.669 − 0.796] for the two test cohorts, respectively; <i>p</i> &lt; 0.05) and Pre-C (AUC: 0.697 [95%CI: 0.554 − 0.819], 0.726 [95%CI: 0.666 − 0.797] for the two test cohorts, respectively; <i>p</i> &lt; 0.05). High AUCs of the MESN-C maintained in the ACRIN 6698 standard (AUC = 0.853 [95%CI: 0.676 − 1.000]) and experimental (AUC = 0.905 [95%CI: 0.817 − 0.993]) subcohorts, and the interracial and external subcohort (AUC = 0.861 [95%CI: 0.811 − 0.906]). Moreover, the MESN-C increased the positive predictive value from 48.6 to 71.3% compared with Pre-C model, and maintained a high negative predictive value (80.4–86.7%).</p> Conclusion <p>The MESN-C using longitudinal multiparametric MRI after a short-term therapy achieved favorable performance for predicting pCR, which could facilitate timely adjustment of treatment regimens, increasing the rates of pCR and avoiding toxic effects.</p> Trial registration <p>Trial registration at <a href="https://www.chictr.org.cn/">https://www.chictr.org.cn/</a>. Registration number: ChiCTR2000038578, registered September 24, 2020.</p>