<p>Emotion recognition is an essential prerequisite for realizing generalized BCI, which possesses an extensive range of applications in real life. EEG-based emotion recognition has become mainstream due to its real-time mapping of brain emotional activities, so a robust EEG-based emotion recognition model is of great interest. However, most existing deep learning emotion recognition methods treat the EEG signal as a whole feature extraction, which will destroy its local stimulation differences and fail to extract local features of the brain region well. Inspired by the cognitive mechanisms of the brain, we propose the multi-brain regions spatiotemporal collaboration transformer (MBRSTCfromer) framework for EEG-based emotion recognition. First, inspired by the prior knowledge, we propose the Multi-Brain Regions Collaboration Network. The EEG data are processed separately after being divided by brain regions, and stimulation scores are presented to quantify the stimulation produced by different brain regions and feedback on the stimulation degree to the MBRSTCfromer. Second, we propose a Cascade Pyramid Spatial Fusion Temporal Convolution Network for multi-brain regions EEG features fusion. Finally, we conduct comprehensive experiments on two mainstream emotion recognition datasets to validate the effectiveness of our proposed MBRSTCfromer framework. We achieved 98.63<InlineEquation ID="IEq1"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation>, 98.15<InlineEquation ID="IEq2"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation>, and 98.58<InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation> accuracy on the three dimensions (arousal, valence, and dominance) on the DEAP dataset; and 97.66<InlineEquation ID="IEq4"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation>, 97.07<InlineEquation ID="IEq5"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation>, and 97.97<InlineEquation ID="IEq6"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation> on the DREAMER dataset.</p>

MBRSTCformer: a knowledge embedded local–global spatiotemporal transformer for emotion recognition

期刊论文
发表日期：

2025年6月17日

查看全文

Chenglin Lin,
Huimin Lu,
Chenyu Pan,
Songzhe Ma,
Zexing Zhang,
Runhui Tian

摘要

Emotion recognition is an essential prerequisite for realizing generalized BCI, which possesses an extensive range of applications in real life. EEG-based emotion recognition has become mainstream due to its real-time mapping of brain emotional activities, so a robust EEG-based emotion recognition model is of great interest. However, most existing deep learning emotion recognition methods treat the EEG signal as a whole feature extraction, which will destroy its local stimulation differences and fail to extract local features of the brain region well. Inspired by the cognitive mechanisms of the brain, we propose the multi-brain regions spatiotemporal collaboration transformer (MBRSTCfromer) framework for EEG-based emotion recognition. First, inspired by the prior knowledge, we propose the Multi-Brain Regions Collaboration Network. The EEG data are processed separately after being divided by brain regions, and stimulation scores are presented to quantify the stimulation produced by different brain regions and feedback on the stimulation degree to the MBRSTCfromer. Second, we propose a Cascade Pyramid Spatial Fusion Temporal Convolution Network for multi-brain regions EEG features fusion. Finally, we conduct comprehensive experiments on two mainstream emotion recognition datasets to validate the effectiveness of our proposed MBRSTCfromer framework. We achieved 98.63 $\%$ $%$ , 98.15 $\%$ $%$ , and 98.58 $\%$ $%$ accuracy on the three dimensions (arousal, valence, and dominance) on the DEAP dataset; and 97.66 $\%$ $%$ , 97.07 $\%$ $%$ , and 97.97 $\%$ $%$ on the DREAMER dataset.