Cross-Modal Coding
摘要
In this chapter, a visual-haptic mutual redundancy elimination scheme is firstly provided by considering mutual assistance between these two modalities. Then, an audio-haptic redundancy elimination mechanism is explored from the perspective of time-frequency masking. Subsequently, by exploring the benefit of potential correlation, a visual-aided haptic coding scheme is proposed to support cross-modal communications. From a theoretical perspective, the minimum number of bits required to compress haptic signals under the rate conditions of visual streams is explored. From a technical perspective, AI-empowered cross-modal prediction and channel coding is designed to support this scheme. Finally, to further achieve the optimum rate of multi-modal streams, a general cross-modal coding strategy is presented, which concerns the mutual assistance between visual and haptic modalities.