<p>An unintelligible video recording of a face uttering a sentence and an unintelligible acoustic sinusoid following the frequency variation of a single vocal resonance of the utterance were intelligible when presented together at their veridical synchrony. The intelligibility resulted from audiovisual sensory integration and phonetic perceptual analysis, which depended neither on the separate resolution of linguistic impressions in each modality nor on closed-set reports about a single pair of minimal phonemic contrast features. Likewise, audiovisual integration could not be attributed to Gestalt-derived similarity principles applied unimodally or bimodally. A single sinusoid lacks natural vocal quality, yet was perceptually coherent with the visible form of a natural articulating face. In other tests, integration also survived desynchronization of the two sensory streams, though only if the visible component led the audible component. These measures expose the robust action and temporal dynamic of early non-phonetic sensory integration which can occur in audiovisual speech perception.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Primitive audiovisual integration of speech

  • Robert E. Remez,
  • Kathryn R. Dubowski,
  • Daria F. Ferro,
  • Emily F. Thomas

摘要

An unintelligible video recording of a face uttering a sentence and an unintelligible acoustic sinusoid following the frequency variation of a single vocal resonance of the utterance were intelligible when presented together at their veridical synchrony. The intelligibility resulted from audiovisual sensory integration and phonetic perceptual analysis, which depended neither on the separate resolution of linguistic impressions in each modality nor on closed-set reports about a single pair of minimal phonemic contrast features. Likewise, audiovisual integration could not be attributed to Gestalt-derived similarity principles applied unimodally or bimodally. A single sinusoid lacks natural vocal quality, yet was perceptually coherent with the visible form of a natural articulating face. In other tests, integration also survived desynchronization of the two sensory streams, though only if the visible component led the audible component. These measures expose the robust action and temporal dynamic of early non-phonetic sensory integration which can occur in audiovisual speech perception.