Scholarlink

<p>Artificial Intelligence has revolutionized emotion recognition, a crucial component in healthcare, security, education, and smart environments. However, traditional approaches relying solely on RGB images suffer from lighting variations, occlusions, noise, and difficulty in capturing subtle emotions. To overcome these limitations, we propose a novel multimodal framework that integrates RGB and thermal images, leveraging physiological cues from temperature to complement visual information. Our method enhances the VGG19 architecture with residual blocks, improving feature representation and robustness. The system is also trained to generate thermal images from RGB inputs, ensuring flexibility when thermal data is unavailable. Experiments conducted on the KDEF dataset demonstrate that our approach achieves 95% classification accuracy across seven emotions: Anger, disgust, fear, happiness, neutrality, sadness, and surprise, outperforming conventional RGB only methods and pretrained models without fine-tuning. These results highlight the effectiveness of combining visible and thermal modalities with deep learning, providing a reliable solution for real-world emotion recognition applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Leveraging deep visual geometry group network for facial emotion recognition through RGB and thermal image fusion

  • 期刊论文

  • 发表日期：

    2026年1月20日

查看全文
  • Tuan-Khoi Tran,
  • Soo-Hyung Kim,
  • Hyung-Jeong Yang,
  • Seung-Won Kim,
  • Ji-Eun Shin

摘要

Artificial Intelligence has revolutionized emotion recognition, a crucial component in healthcare, security, education, and smart environments. However, traditional approaches relying solely on RGB images suffer from lighting variations, occlusions, noise, and difficulty in capturing subtle emotions. To overcome these limitations, we propose a novel multimodal framework that integrates RGB and thermal images, leveraging physiological cues from temperature to complement visual information. Our method enhances the VGG19 architecture with residual blocks, improving feature representation and robustness. The system is also trained to generate thermal images from RGB inputs, ensuring flexibility when thermal data is unavailable. Experiments conducted on the KDEF dataset demonstrate that our approach achieves 95% classification accuracy across seven emotions: Anger, disgust, fear, happiness, neutrality, sadness, and surprise, outperforming conventional RGB only methods and pretrained models without fine-tuning. These results highlight the effectiveness of combining visible and thermal modalities with deep learning, providing a reliable solution for real-world emotion recognition applications.