Leveraging LLM-Generated Data for Detecting Depression Symptoms on Social Media

In our work, we present the contribution of the BLUE team in the eRisk Lab task focused on identifying symptoms of depression in Reddit social media posts. The task consists of retrieving and ranking Reddit social media sentences that convey symptoms of depression from the BDI-II questionnaire. To augment our data and improve downstream models, we utilized synthetic data generated by GPT-3.5 and LLama-3 for each of the BDI-II symptoms. Our approach aimed to enrich the data with semantic diversity and emotional and anecdotal experiences that are specific to the more intimate way of sharing experiences on Reddit. We used semantic search and cosine similarity to rank the relevance of the sentences to the BDI-II symptoms. Our study compared the performance of two transformer-based models (MentalRoBERTa and a variant of MPNet) in embedding social media posts and the original/generated BDI-II responses for information retrieval. We found that using sentence embeddings from a model designed for semantic search outperformed the approach using embeddings from a model pre-trained on mental health data. Furthermore, the generated synthetic data were proved too specific for this task, the approach simply relying on the BDI-II responses had the best performance.