This study introduces a novel framework for enhancing the dynamic interactions between players and Virtual Humans (VH) in immersive environments. While we mainly focus on video games, this work has various applications in other fields, such as marketing, health, fiction, and robotics. Leveraging recent advances in Artificial Intelligence (AI) and Natural Language Processing (NLP), the system generates contextually responsive dialogue and adaptive emotional behaviors using Large Language Models (LLM). The system is designed with multiple interrelated components that collectively enable a seamless integration of personality expression, dialogue generation, emotion simulation, and real-time visual rendering, through an experimental communication protocol with Gemini. Experimental evaluations conducted in both Virtual Reality (VR) and non-VR settings indicate a generally positive reception, with participants reporting innovative character credibility, enhanced emotional expressiveness, and heightened overall immersion. The results underscore the potential of AI-driven VHs to overcome the limitations of traditional scripted systems, thereby enriching narrative engagement and interactive experiences. Future work will focus on further refining emotion recognition and extending the framework to support increasingly complex interactive scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Framework for Enhancing Emotion Expression in Non-Playable Characters Using Large Language Models

  • Marco Ligabue,
  • Susanna Brambilla,
  • Laura Anna Ripamonti,
  • Francesco Bultrini,
  • Andrea Zaniboni,
  • Nunzio Alberto Borghese

摘要

This study introduces a novel framework for enhancing the dynamic interactions between players and Virtual Humans (VH) in immersive environments. While we mainly focus on video games, this work has various applications in other fields, such as marketing, health, fiction, and robotics. Leveraging recent advances in Artificial Intelligence (AI) and Natural Language Processing (NLP), the system generates contextually responsive dialogue and adaptive emotional behaviors using Large Language Models (LLM). The system is designed with multiple interrelated components that collectively enable a seamless integration of personality expression, dialogue generation, emotion simulation, and real-time visual rendering, through an experimental communication protocol with Gemini. Experimental evaluations conducted in both Virtual Reality (VR) and non-VR settings indicate a generally positive reception, with participants reporting innovative character credibility, enhanced emotional expressiveness, and heightened overall immersion. The results underscore the potential of AI-driven VHs to overcome the limitations of traditional scripted systems, thereby enriching narrative engagement and interactive experiences. Future work will focus on further refining emotion recognition and extending the framework to support increasingly complex interactive scenarios.