Objectives <p>The primary aim of this study was to quantify the subject-level test–retest repeatability of artificial intelligence (AI)-derived PSMA PET imaging biomarkers using a previously developed model. The secondary aim was to assess the performance of this segmentation model, which was trained on [⁶⁸&#xa0;Ga]Ga-PSMA-11 PET scans, on [¹⁸F]F-PSMA-1007 PET scans.</p> Methods <p>This was a post-hoc analysis of a prospective, single-center, test–retest trial. Seventeen patients with metastatic prostate cancer (mPCa) were randomised into groups, either receiving the same tracer ([⁶⁸&#xa0;Ga]Ga-PSMA-11 or [¹⁸F]F-PSMA-1007) for both scans (intra-tracer group, n = 9) or a different tracer (inter-tracer group, n = 8). Scans were delineated using a fully automated AI method and semi-automatically. The subject-level repeatability of four imaging biomarkers, including PSMA-positive tumour volume, was quantified.</p> Results <p>Repeatability analysis demonstrated poorer repeatability for all biomarkers in the inter-tracer group. In the intra-tracer group, the AI-derived PSMA-positive tumour volume had a repeatability coefficient of 13.8% for higher volume disease patients (≥ median tumour volume). There was no significant difference in the per-scan lesion-level positive predictive value of the AI model between [⁶⁸&#xa0;Ga]Ga-PSMA-11 and [¹⁸F]F-PSMA-1007 PET scans (0.88, IQR 0.69—1.00 vs. 0.78, IQR 0.54—1.00, p = 0.60).</p> Conclusion <p>AI-based PSMA-positive tumour volume calculations have repeatability limits that are consistent with the use of the Response Evaluation Criteria in PSMA PET/CT (RECIP 1.0) criteria for higher volume disease patients when the same tracer is used. Substantially wider repeatability limits in the inter-tracer group provide evidence that response assessment should be conducted using the same radiotracer.</p>