The concept of sXAI is inspired by (human) social interaction, the primary case of which is face-to-face communication involving not only speech and language but also nonverbal modalities such as facial expression, gaze, gesture, head movement, and body posture. Such multimodal communication facilitates language processing in humans and becomes relevant for sXAI systems when they are (embodied) communicating agents able to produce multimodal behaviors and understand those of human users. In this chapter, we describe theoretical aspects of multimodal processing in human interaction (binding and segregation of modalities, how multimodal behavior is produced and understood) and their relation to computational models of multimodal interaction for conversational agents.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Theoretical Aspects of Multimodal Processing

  • Angela Grimminger,
  • Hendrik Buschmeier

摘要

The concept of sXAI is inspired by (human) social interaction, the primary case of which is face-to-face communication involving not only speech and language but also nonverbal modalities such as facial expression, gaze, gesture, head movement, and body posture. Such multimodal communication facilitates language processing in humans and becomes relevant for sXAI systems when they are (embodied) communicating agents able to produce multimodal behaviors and understand those of human users. In this chapter, we describe theoretical aspects of multimodal processing in human interaction (binding and segregation of modalities, how multimodal behavior is produced and understood) and their relation to computational models of multimodal interaction for conversational agents.