Functional magnetic resonance imaging (fMRI) has the capability to reflect brain activities, with brain regions such as the amygdala, prefrontal cortex, and hippocampus exhibiting distinct emotional activity patterns. These neural activities interact in complex ways, making interpretation and identification challenging. The objective of this study is to identify participants’ emotions using fMRI data from the ICBHI scientific challenge, involving 16 participants in the training set and 4 in the test set. Each participant has undergone 30 trails and reported their emotion classes and levels. The averaged fMRI signals were extracted from 246 regions by Brainnetome atlas, and the regions of interest were selected based on correlations, including emotional related 66 regions. We applied a band-pass filter (sigma = 2) for data preprocessing. There were two models developed, for class and level prediction, respectively. For class prediction, to enhance signal quality and reduce noise in fMRI signals, we employed a nonlinear least squares approach. Specifically, each 25-s signal segment was modeled using a polynomial, and parameters of this polynomial were adjusted to minimize the sum of squares of the differences between observed and predicted values. To test which brain regions’ activity patterns are highly correlated with emotions, we trained multiple fully connected neural networks (FCNNs) to select from the previously mentioned 66 brain regions. Ultimately, the signals from brain regions highly correlated with emotions are selected as inputs for subsequent integrated convolutional neural network (CNN) model to distinguish between types of emotions. For level prediction, we utilized long short- term memory (LSTM) models for level prediction to participants’ emotions judgement. Finally, the algorithm we proposed achieved a classification accuracy of 56% for identifying emotion classes, 20% for identifying emotional intensity levels. The mean score of 0.6251 was calculated by ICBHI. The difference of the precision for class and level may be related to the subjective evaluation of the subjects as the rating of emotion is highly dependent by the subject’s self-perception and background.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhanced Emotion Recognition from fMRI Using a Multi-model Neural Network Framework

  • Mi-Hsuan Lin,
  • Pin-Han Chen,
  • Rong-Huan Huang,
  • Tzu-Yao Lo,
  • Li-Yun Tseng,
  • Zi-Xiang Tsai,
  • Chun-Yi Zac Lo

摘要

Functional magnetic resonance imaging (fMRI) has the capability to reflect brain activities, with brain regions such as the amygdala, prefrontal cortex, and hippocampus exhibiting distinct emotional activity patterns. These neural activities interact in complex ways, making interpretation and identification challenging. The objective of this study is to identify participants’ emotions using fMRI data from the ICBHI scientific challenge, involving 16 participants in the training set and 4 in the test set. Each participant has undergone 30 trails and reported their emotion classes and levels. The averaged fMRI signals were extracted from 246 regions by Brainnetome atlas, and the regions of interest were selected based on correlations, including emotional related 66 regions. We applied a band-pass filter (sigma = 2) for data preprocessing. There were two models developed, for class and level prediction, respectively. For class prediction, to enhance signal quality and reduce noise in fMRI signals, we employed a nonlinear least squares approach. Specifically, each 25-s signal segment was modeled using a polynomial, and parameters of this polynomial were adjusted to minimize the sum of squares of the differences between observed and predicted values. To test which brain regions’ activity patterns are highly correlated with emotions, we trained multiple fully connected neural networks (FCNNs) to select from the previously mentioned 66 brain regions. Ultimately, the signals from brain regions highly correlated with emotions are selected as inputs for subsequent integrated convolutional neural network (CNN) model to distinguish between types of emotions. For level prediction, we utilized long short- term memory (LSTM) models for level prediction to participants’ emotions judgement. Finally, the algorithm we proposed achieved a classification accuracy of 56% for identifying emotion classes, 20% for identifying emotional intensity levels. The mean score of 0.6251 was calculated by ICBHI. The difference of the precision for class and level may be related to the subjective evaluation of the subjects as the rating of emotion is highly dependent by the subject’s self-perception and background.