Scholarlink

<p>Graph neural networks (GNNs) have demonstrated strong performance in handling graph-structured data in recent years, particularly in capturing complex inter-node relationships among data samples, showcasing advantages over traditional neural networks. However, challenges persist, including difficulties in cross-modal information fusion, inadequate modeling of modal relationships, and high computational costs. To address these limitations, this paper proposes GGMEN, a novel model that integrates the local neighborhood aggregation capability of graph convolutional networks with the global structural expressiveness of graph isomorphic networks (GINs). Leveraging shallow feature extraction via time-frequency joint analysis, the paper extracts 14 representative physiological statistical features. Simultaneously, the Transformer model captures spatial features from individual facial expression video frames, enabling spatio-temporal modeling of facial expressions. The graph convolutional network (GCN) layer models temporal dependencies in physiological signals and spatial relationships of facial key points, while the GIN layer enhances modeling of complex higher-order relationships. Multi-modal emotion perception is achieved through attention-based modality fusion. Experiments on multiple datasets validate the model’s effectiveness across multiple emotion perception benchmarks, achieving an emotion recognition accuracy of 81.25% on dataset DEAP. Comparative analyses with existing models confirm the accuracy improvement of the proposed framework.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Research on a multimodal emotion perception model based on GCN+GIN hybrid model

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月23日

查看全文
  • Yingqiang Wang,
  • Elcid A. Serrano

摘要

Graph neural networks (GNNs) have demonstrated strong performance in handling graph-structured data in recent years, particularly in capturing complex inter-node relationships among data samples, showcasing advantages over traditional neural networks. However, challenges persist, including difficulties in cross-modal information fusion, inadequate modeling of modal relationships, and high computational costs. To address these limitations, this paper proposes GGMEN, a novel model that integrates the local neighborhood aggregation capability of graph convolutional networks with the global structural expressiveness of graph isomorphic networks (GINs). Leveraging shallow feature extraction via time-frequency joint analysis, the paper extracts 14 representative physiological statistical features. Simultaneously, the Transformer model captures spatial features from individual facial expression video frames, enabling spatio-temporal modeling of facial expressions. The graph convolutional network (GCN) layer models temporal dependencies in physiological signals and spatial relationships of facial key points, while the GIN layer enhances modeling of complex higher-order relationships. Multi-modal emotion perception is achieved through attention-based modality fusion. Experiments on multiple datasets validate the model’s effectiveness across multiple emotion perception benchmarks, achieving an emotion recognition accuracy of 81.25% on dataset DEAP. Comparative analyses with existing models confirm the accuracy improvement of the proposed framework.