We propose methods for training emotion recognition models using VR facial expression data (52 blendshape parameters). We evaluate linear models (linear regression and support vector regression), logistic regression, a neural network, and their ensemble. Linear models are trained to predict continuous Valence-Arousal-Dominance (VAD) values from inputs, and final emotions are assigned by finding the nearest emotion point in VAD space. Logistic regression and the neural network directly classify into 7 emotion classes. We present the neural network architecture with three hidden layers of 64 neurons each and a softmax output. Models are trained with the Adam optimizer, and we evaluate the correlation of predicted VAD with true values. In experiments, logistic regression achieved ~60% overall accuracy with high VAD correlations (around 0.64–0.68), while the neural network reached 56% accuracy with moderate correlation (Valence 0.517, Arousal 0.433, Dominance 0.671). Linear models were much lower (~29–44% accuracy), with SVM clearly outperforming simple regression. An ensemble of three logistic models gave 47% accuracy, and three neural networks gave 54%, both below the best single models. We also describe an inverse mapping model from VAD to avatar morph target parameters. We discuss failure cases (frequent confusion of fear and surprise, over-prediction of neutral emotion) and the need for a larger dataset. The proposed approaches demonstrate the applicability of VR-based emotion recognition for social interaction systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Emotion Recognition from Facial Expression Using VR Headset

  • Anatoly Dolgikh,
  • Eduard Radostev

摘要

We propose methods for training emotion recognition models using VR facial expression data (52 blendshape parameters). We evaluate linear models (linear regression and support vector regression), logistic regression, a neural network, and their ensemble. Linear models are trained to predict continuous Valence-Arousal-Dominance (VAD) values from inputs, and final emotions are assigned by finding the nearest emotion point in VAD space. Logistic regression and the neural network directly classify into 7 emotion classes. We present the neural network architecture with three hidden layers of 64 neurons each and a softmax output. Models are trained with the Adam optimizer, and we evaluate the correlation of predicted VAD with true values. In experiments, logistic regression achieved ~60% overall accuracy with high VAD correlations (around 0.64–0.68), while the neural network reached 56% accuracy with moderate correlation (Valence 0.517, Arousal 0.433, Dominance 0.671). Linear models were much lower (~29–44% accuracy), with SVM clearly outperforming simple regression. An ensemble of three logistic models gave 47% accuracy, and three neural networks gave 54%, both below the best single models. We also describe an inverse mapping model from VAD to avatar morph target parameters. We discuss failure cases (frequent confusion of fear and surprise, over-prediction of neutral emotion) and the need for a larger dataset. The proposed approaches demonstrate the applicability of VR-based emotion recognition for social interaction systems.