Hybrid BERT-XGBoost transformer models for predicting mental health from social media
摘要
The increasing use of social media has provided a platform for self-expression, but has also been linked to rising mental health challenges such as depression, anxiety, and suicidal ideation. Early detection of these conditions is crucial for timely intervention and prevention of suicide. This study proposes a hybrid deep learning model that integrates transformer-based architectures with XGBoost for automatic mental health prediction from social media content. We analyzed textual data from Facebook, Reddit, and Twitter to classify users’ mental health statuses into seven categories: Normal, Depression, Suicidal, Anxiety, Stress, Bipolar, and Personality Disorder. We compared traditional machine learning models (Naïve Bayes, Decision Tree, Logistic Regression, SVM, and XGBoost) with deep transformer-based models (BERT, RoBERTa, and DistilBERT). The best performing transformer models were then combined with XGBoost to improve classification accuracy. XGBoost outperformed traditional classifiers, while RoBERTa achieved the highest accuracy among transformer models. The hybrid BERT-XGBoost and RoBERTa-XGBoost models further improved performance, achieving 94% accuracy, surpassing both standalone transformers and traditional classifiers. This confirms the effectiveness of combining the extraction of contextual features from transformers with the efficient classification capabilities of XGBoost. This study highlights the effectiveness of hybrid transformer-XGBoost models in detecting mental health conditions from social media data. By combining the powerful feature extraction capabilities of transformers with the efficiency of XGBoost, our approach enhances both diagnostic accuracy and robustness. The results obtained confirm the potential of this hybrid model for reliable and automated mental health prediction, which contributes to more effective early intervention strategies.