Bilingual speech separation presents significant challenges and opportunities in the field of speech processing. Vietnamese and English differ fundamentally in their phonetic structures. Vietnamese is a tonal language, while English relies on stress patterns and non-tonal distinctions. This disparity complicates the separation of mixed-language speech, as traditional models may struggle to accurately distinguish overlapping phonemes and words from the two languages. The MossFormer2 is an advanced model combining Transformer-based self-attention mechanisms with a feedforward sequential memory network, enhances the ability to model both long-range dependencies and fine-scale recurrent patterns in speech signals. This hybrid approach improves the separation of mixed-language audio by effectively handling the unique acoustic characteristics of both Vietnamese and English. By leveraging this model, researchers achieve more accurate separation of bilingual speech mixtures, facilitating clearer transcription and analysis in multilingual environments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Bilingual Code-Switching Voice Separation with MossFormer2 for Vietnamese-English Mixtures

  • Pham Manh Kha,
  • Le Duong Tan Minh,
  • Ha Minh Tan,
  • Diem Thi Tran,
  • Duc-Quang Vu

摘要

Bilingual speech separation presents significant challenges and opportunities in the field of speech processing. Vietnamese and English differ fundamentally in their phonetic structures. Vietnamese is a tonal language, while English relies on stress patterns and non-tonal distinctions. This disparity complicates the separation of mixed-language speech, as traditional models may struggle to accurately distinguish overlapping phonemes and words from the two languages. The MossFormer2 is an advanced model combining Transformer-based self-attention mechanisms with a feedforward sequential memory network, enhances the ability to model both long-range dependencies and fine-scale recurrent patterns in speech signals. This hybrid approach improves the separation of mixed-language audio by effectively handling the unique acoustic characteristics of both Vietnamese and English. By leveraging this model, researchers achieve more accurate separation of bilingual speech mixtures, facilitating clearer transcription and analysis in multilingual environments.