In this chapter, the typical multi-modal services that have emerged in recent years are firstly described. These services can be categorized into three main types: haptic-dominant, audio-visual-dominant, and equally important multi-modal services. Following this, an analysis of the characteristics of different modalities is conducted. On the one hand, the heterogeneous communication requirements of audio, visual, and haptic signals are examined. On the other hand, the diverse communication requirements of the different categories of multi-modal services are also presented. Subsequently, to meet these requirements, a cross-modal communication architecture is constructed and the details of its components are described. Finally, the key techniques within this architecture are introduced. Additionally, the issues that need to be addressed as well as the core ideas behind the design of these key techniques are also discussed.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Cross-Modal Communication Architecture

  • Xin Wei,
  • Dan Wu,
  • Liang Zhou

摘要

In this chapter, the typical multi-modal services that have emerged in recent years are firstly described. These services can be categorized into three main types: haptic-dominant, audio-visual-dominant, and equally important multi-modal services. Following this, an analysis of the characteristics of different modalities is conducted. On the one hand, the heterogeneous communication requirements of audio, visual, and haptic signals are examined. On the other hand, the diverse communication requirements of the different categories of multi-modal services are also presented. Subsequently, to meet these requirements, a cross-modal communication architecture is constructed and the details of its components are described. Finally, the key techniques within this architecture are introduced. Additionally, the issues that need to be addressed as well as the core ideas behind the design of these key techniques are also discussed.