Depression Detection by Extracting PHQ-8 Scores from Semi-clinical Interviews Using Machine Learning and Deep Learning
摘要
Depression stands as a major concern within the domain of mental health, underscoring the critical importance of assessing its severity. Timely detection and appropriate treatment of severe depression can significantly mitigate the risk of fatalities among affected individuals. Mental health professionals, including psychiatrists and psychologists, often conduct clinical interviews comprising structured yet simple conversations to determine depression levels in patients. Additionally, standardized psychometric questionnaires like PHQ-8 offer a quantifiable score directly correlating with depression severity. Unfortunately, the availability of psychiatrists and psychologists proficient in diagnosing and treating such cases remains limited. To tackle these challenges, this research work proposes the automation of the PHQ-8 questionnaire through artificial intelligence and Natural Language Processing (NLP) techniques, leveraging semi-clinical interviews with patients. Various machine learning (Ridge Regression, Random Forest, XGBoost, LightGBM, CatBoost) and deep learning methods (GRU and LSTMs) are evaluated and compared using textual transcripts sourced from the Extended Distress Analysis Interview Corpus (E-DAIC). Moreover, the proposed models demonstrate superior performance (lower RMSE) compared to baseline results as provided by the AVEC 2019 initiative utilizing the same dataset for the ‘Detecting Depression with AI’ sub-challenge.