Background <p>Current National Comprehensive Cancer Network guidelines define brachytherapy monotherapy as a suitable treatment for favorable intermediate risk (FIR) and unfavorable intermediate risk (UIR) prostate cancer. Our objective is to define the subgroup of patients suitable for brachytherapy monotherapy.</p> Methods <p>We conducted a single-institutional retrospective analysis of intermediate risk prostate cancer, treated with brachytherapy with or without androgen deprivation therapy (ADT) and/or external beam radiation therapy (EBRT). The primary endpoint was biochemical failure (BF), defined as prostate specific antigen (PSA) &gt; 0.4 ng/mL. For monotherapy, multivariate Fine-Gray analysis was used to identify risk factors associated with BF. Univariate analysis was performed to evaluate whether ADT and/or EBRT were associated with BF for patients without and with such factors.</p> Results <p>Among 1622 patients, the median follow-up was 10.4 years. For monotherapy, PSA ≥ 10 ng/mL (adjusted sHR 3.01; 95% CI: 1.10–8.27; <i>p</i> = 0.032) and cT2b-c disease (adjusted sHR 4.52; 95% CI: 1.85–11.07; <i>p</i> = 0.001) were associated with BF. The 10-year incidences of BF after monotherapy for patients without and with these risk factors were 5.8% (3.8% FIR, 8.8% UIR) versus 17.2% (9.3% FIR, 23.9% UIR), respectively. For the cT1-T2a/PSA &lt; 10 risk group, neither the addition of ADT (sHR 0.90; 95% CI: 0.38–2.1; <i>p</i> = 0.82) nor EBRT (sHR 0.65; 95% CI: 0.36-1.18; <i>p</i> = 0.16) was associated with biochemical failure. For the cT2b-T2c and/or PSA ≥ 10 subgroup, ADT (sHR: 0.33; 95% CI: 0.14–0.74; <i>p</i> = 0.007) but not EBRT (sHR 0.66; 95% CI: 0.34–1.31; <i>p</i> = 0.24) was associated with BF.</p> Conclusions <p>Brachytherapy monotherapy is suitable for all FIR, and UIR disease meeting cT1-T2a/PSA &lt; 10 criteria.</p>