A Resilient Overlay for Human Emotion Recognition Using Mixed Frameworks in Machine-Human Interactions

Human sentiments are irrational responses to things or situations associated with various physiological, behavioural, and cognitive processes. Understanding human sentiments is pivotal in diverse fields such as human–computer interaction, virtual reality, self-driving technology, and health monitoring. The scientific community is becoming more and more interested in emotion recognition. Emotion recognition, mainly through brain-generated electroencephalogram (EEG) waves, has gained significant attention in developing brain-computer interfaces. This article addresses critical research gaps in existing cross-subject emotion detection methods based on EEG signals and multi-modal techniques that combine EEG data with other modalities. We proposed an effective hybrid technique for cross-subject sentiment identification using EEG and facial movements. The suggested method combines a spectrum of directed variations and local binary pattern characteristics with spectral and statistical features from the facial dataset. The next step is classifying emotions using support vector machines, k-nearest neighbour, and ensembles. Notably, our method tackles the challenge of category imbalance using an up-sampling approach. Our results are promising after evaluating the efficiency of our proposed technique on a sentiment analysis dataset with physiological data and employing tenfold cross-validation. Notably, the accuracy levels for valence and arousal reach impressive highs at 96.25% and 97.10%, respectively.