Enhancing Machine Translation Performance: A Comparative Study of Finetuning the MBart-mmt Model
摘要
This study examines how well the fine-tuned mbart-mmt (mbart-large-50-many-to-many-mmt) model, the unfinetuned mbart-mmt model, and the Helsinki-NLP/opus-mt model perform in translating the Malayalam language. We used the ‘Samanantar’ dataset, which initially contained poor-quality English-Malayalam translations. To address this issue, we employed Language-agnostic BERT Sentence Embedding (LaBSE) to filter the dataset. Specifically, we generated sentence embeddings for both the English and Malayalam languages, calculated the cosine similarity between these embeddings, and filtered the dataset using a threshold value of 0.91 to retain only high-quality translations. Additionally, we augmented the training data with manually translated datasets and datasets from ‘IN22-Conv,’ ‘IN22-Gen,’ and the ‘dev’ subset of the ‘flores200’ dataset. The evaluation metrics used were the BLEU and chrF scores. The fine-tuned model achieved a average BLEU score of 26.2 and a chrF score of 53.9 on the test data, indicating significantly better performance compared to the base model, which had a low BLEU score of 8 and a chrF score of 28.5.