Depression is a common mental health disorder worldwide, where heterogeneous presentations, high psychiatric comorbidity, and the episodic nature of this disorder compound clinical diagnosis difficulty. Social stigma combined with difficulty accessing mental health professionals further compounds this issue. This review investigates the potential of using explainable AI (xAI) as a diagnostic tool to help increase the quality-of-service delivery to patients. Through a structured literature extraction process via five prominent electronic databases, and following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we screened a total of 62 scientific papers, resulting in 17 eligible articles for an in-depth analysis. A detailed thematic analysis presents the main areas of trust, model choice, model training, data utilisation, and personalisation, along with the key metrics/techniques used in the latest research. We found transformer-based models using textual data such as user-generated content in a journal or social media format offer robust categorisation performance of a user’s mental health status. xAI techniques are well developed for feature engineering and interpretation. However, further work is required to develop robust user explainability and understandability as well as in terms of presentation in a clinically relevant manner.