Deep Learning Approaches for Emotion Recognition with Missing PPG and RESP Signals Using Multi-channel Data
摘要
The 2024 ICBHI Challenge requires the design of an effective method to analyze three different emotions and their intensity levels. The data is recorded using three different instruments to capture the participants’ emotions while watching videos: functional magnetic resonance imaging (fMRI), photoplethysmography (PPG), and respiratory (RESP) signals. However, there are missing PPG and RESP signals, resulting in multiple emotion records that only have fMRI signals. Most methods may use only fMRI as training data to build models, but this would waste the advantage of using PPG and RESP signal features to assist the model in achieving more efficient discrimination. Therefore, this study proposes effective methods for compensating for missing PPG and RESP data, correcting offset values, and effectively augmenting fMRI data. These methods address the aforementioned issues to construct a multi-channel deep learning model based on the three types of data. The model proposed in this study achieved an overall error rate of only 0.2844 on the official evaluation metrics, indicating that the proposed method can achieve high-performance emotion and emotion intensity discrimination.