Delineating emotional differences between depressed and non-depressed individuals using a novel multimodal framework

Affective computing has garnered substantial attention for its potential in mental health diagnostics, including the nuanced differentiation of emotional states in depressed versus non-depressed individuals. This study significantly advances the field by introducing EmoFusion-AttNet, a state-of-the-art computational model that employs a multimodal approach for emotion classification. Leveraging a statistically robust and demographically diverse sample, participants were first screened for depressive symptoms using BDI & PHQ-Q questionnaires, ensuring a representative distribution. The EmoFusion-AttNet model employs Convolutional Neural Networks (CNNs) to extract spatial features from facial data, while Long Short-Term Memory (LSTM) networks capture the temporal intricacies of EEG signals. A fusion layer equipped with an attention mechanism synthesizes these multi-dimensional data streams, enhancing feature selection and accuracy. Impressively, the model achieved a classification accuracy of 95%, setting a new benchmark for emotion-based classification models. To validate these findings, we conducted an in-depth analysis of EEG data, focusing on Alpha and Theta band activities. Our results revealed a decrease in Alpha power and an increase in Theta power in the depressed group, aligning with current neurophysiological understandings. These findings validate the model’s clinical applicability and set the stage for future research in affective computing, particularly its implications for mental health diagnostics and treatment planning.