Multimodal Emotion Classification: Implications for Cognitive Science and Human Behaviour

Emotions play a crucial role in our daily lives, influencing our behavior, thoughts, and decisions. With the advancements in artificial intelligence and machine learning (AIML), it is now possible to classify emotions based on physiological signals, such as heart rate, skin conductance, and electroencephalography (EEG). We aim to investigate the effectiveness of using physiological signals for emotional recognition in this research paper. This study explores different feature extraction, feature selection, and classification techniques for emotion recognition using physiological sensors available in the MAHNOB-HCI dataset, a publicly available benchmark dataset widely used in affective computing. A number of machine learning techniques are employed to analyze unimodal and multimodal implementations on modalities such as EEG, Electrocardiogram (ECG), and Galvanic Skin Response (GSR) and performed binary and 3-class classification. Compared to recent studies which have shown that deep learning-based multimodal approaches can achieve superior performance in emotion recognition tasks as compared to machine learning algorithms, with classification rates of 99.2% for valence and 99.3% for arousal using machine learning algorithms.