Talk Mirror is a fluent and quick multilingual speech translation program aimed to university students and other people, who have a need in having some transparency when it comes to voice replication. With Whisper and mBART, the system does safe speech-to-text transcriptions, safe context-sensitive neural machine translation, and natural-speech text-to-speech translation. To preserve the original voice quality of the speaker Parselmouth and Librosa are used to extract pitch, formants and speaking rate. Additional characteristics like flexible noise cancellation and regular formant saving increase the expression of speech and suppress distortion of the original acoustic signature created by an individual. This has resulted in the system obtaining high-fidelity voice cloning in the presence of a wide range of acoustic conditions. The concept behind it, as demonstrated by the idea of reflecting speech into a different language without losing the tone of the speaker and his spirit, is reflected in the name Talk Mirror. Talk Mirror is the actual reflection of voice, speech and emotions of a human person just like mirror brings out the appearance of an individual.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Talk Mirror Language to Language Translation with Voice Preservation

  • K. M. Kiruthick,
  • S. Rakesh,
  • K. Prajesh,
  • C. Guru Sai Sree,
  • N. Neelima,
  • T. Glace Varghese

摘要

Talk Mirror is a fluent and quick multilingual speech translation program aimed to university students and other people, who have a need in having some transparency when it comes to voice replication. With Whisper and mBART, the system does safe speech-to-text transcriptions, safe context-sensitive neural machine translation, and natural-speech text-to-speech translation. To preserve the original voice quality of the speaker Parselmouth and Librosa are used to extract pitch, formants and speaking rate. Additional characteristics like flexible noise cancellation and regular formant saving increase the expression of speech and suppress distortion of the original acoustic signature created by an individual. This has resulted in the system obtaining high-fidelity voice cloning in the presence of a wide range of acoustic conditions. The concept behind it, as demonstrated by the idea of reflecting speech into a different language without losing the tone of the speaker and his spirit, is reflected in the name Talk Mirror. Talk Mirror is the actual reflection of voice, speech and emotions of a human person just like mirror brings out the appearance of an individual.