The Tulu-to-English speech-to-speech translation system bridges language barriers by converting spoken Tulu phrases into fluent English speech through an end-to-end pipeline. The system integrates automatic speech recognition (ASR) for Tulu speech transcription, a sequence-to-sequence translation model for Tulu-to-English text conversion, and text-to-speech (TTS) synthesis for generating natural-sounding English speech. The model employs a sequential architecture consisting of an embedding layer (256-dimensional), an LSTM layer to capture sequential dependencies, and a time distributed dense layer with softmax activation for word prediction. The model is trained on 3000 datasets over 150 epochs, achieving an accuracy of 78% and a loss of 1%. Optimization is performed using the Adam optimizer and sparse categorical cross-entropy loss function, with a BLEU score of 64.35%, reflecting strong semantic and syntactic alignment, and a word error rate (WER) of 14.28%, suggesting areas for further improvement in word-level accuracy.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Speech-to-Speech Translation for Unwritten Language

  • Sudheeksha,
  • Dishika K. Kanchan,
  • C. Shreya,
  • Aksha S. Kunder,
  • Raghavendra Sooda

摘要

The Tulu-to-English speech-to-speech translation system bridges language barriers by converting spoken Tulu phrases into fluent English speech through an end-to-end pipeline. The system integrates automatic speech recognition (ASR) for Tulu speech transcription, a sequence-to-sequence translation model for Tulu-to-English text conversion, and text-to-speech (TTS) synthesis for generating natural-sounding English speech. The model employs a sequential architecture consisting of an embedding layer (256-dimensional), an LSTM layer to capture sequential dependencies, and a time distributed dense layer with softmax activation for word prediction. The model is trained on 3000 datasets over 150 epochs, achieving an accuracy of 78% and a loss of 1%. Optimization is performed using the Adam optimizer and sparse categorical cross-entropy loss function, with a BLEU score of 64.35%, reflecting strong semantic and syntactic alignment, and a word error rate (WER) of 14.28%, suggesting areas for further improvement in word-level accuracy.