<p>Machine translation methods help bridge the language gap worldwide, making communication easier, supporting automatic translation, and enabling faster convergence than a human translator. However, machine translation methods are data-driven, and the quality of translation depends on the availability of datasets or the number of parallel corpora. In recent years, neural methods have witnessed growth in almost every field, including industrial and academic research. Despite limited resources, neural machine translation methods are progressively achieving better accuracy than other existing methods. Taking advantage of neural methods, we aim to explore the challenges of using the low-resource language Nyishi. To implement and assess the effectiveness of the bilingual evaluation understudy (BLEU) score, we used a neural-based transformer model from Nyishi to English. Additional byte-pair encoding (BPE) tokenization methods are applied to evaluate model performance and address the rare-word problem in a limited-resource environment. The BLEU score with BPE tokenization improves the translation accuracy by 2% in transformer and neural machine translation (NMT) with the global attention model. Finally, the effectiveness of translations predicted by the trained models has been evaluated using both automatic and human evaluation metrics. We also looked at how well the models predicted based on the length of sentence.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating the effectiveness of neural machine translation from Nyishi-to-English

  • Nabam Kakum,
  • Rushanti Kri,
  • Koj Sambyo

摘要

Machine translation methods help bridge the language gap worldwide, making communication easier, supporting automatic translation, and enabling faster convergence than a human translator. However, machine translation methods are data-driven, and the quality of translation depends on the availability of datasets or the number of parallel corpora. In recent years, neural methods have witnessed growth in almost every field, including industrial and academic research. Despite limited resources, neural machine translation methods are progressively achieving better accuracy than other existing methods. Taking advantage of neural methods, we aim to explore the challenges of using the low-resource language Nyishi. To implement and assess the effectiveness of the bilingual evaluation understudy (BLEU) score, we used a neural-based transformer model from Nyishi to English. Additional byte-pair encoding (BPE) tokenization methods are applied to evaluate model performance and address the rare-word problem in a limited-resource environment. The BLEU score with BPE tokenization improves the translation accuracy by 2% in transformer and neural machine translation (NMT) with the global attention model. Finally, the effectiveness of translations predicted by the trained models has been evaluated using both automatic and human evaluation metrics. We also looked at how well the models predicted based on the length of sentence.