Automated Flagging of Cognitive Biases in the Spoken Language of People with Hallucination Experiences
摘要
Cognitive biases associated with psychotic symptoms, including auditory verbal hallucinations (AVH), can predict poor outcomes. Traditional assessment of AVH often relies on retrospective reporting, prone to bias and inaccuracies. We test four natural language processing (NLP) strategies to identify cognitive biases in audio diaries collected from people experiencing AVH. Our models categorize thinking at three levels: (1) any cognitive bias (binary), (2) thinking style, and (3) cognitive distortion. We hypothesize models will be best for binary, followed by thinking style, and then specific distortion classification. Audio diaries were collected from 295 individuals with AVH over 30 days using smartphones. We transcribed and randomly selected diaries for annotation (N = 511) of biases by psychologists trained in CBT for psychosis. We implement three BERT models along with one large language model (LLM) to achieve classification tasks and evaluate performance using F1 score. A BERT model with transfer learning achieved the best performance across most classification tasks, and all BERT models significantly outperformed LLM. The models performed highest for binary classification (F1 = 0.61), followed by classification of thinking styles and cognitive distortions. Performance improved on labels with low representation using transfer learning and grouping distortions into higher-order thinking styles. NLP methods effectively identified cognitive biases in audio diaries from individuals with AVH. Classifying at the higher-order level of thinking style compared to lower-order cognitive distortions improves performance. Fine-tuned BERT models perform better than LLM. The technology demonstrated in this study can advance personalized, scalable assessment and intervention for symptoms of psychosis.