Mongolian-Chinese Neural Machine Translation Based on Sustained Transfer Learning

In neural machine translation (NMT), Mongolian-Chinese NMT (MNMT) encounters significant challenges due to the limited volume and accessibility of parallel corpus data. This leads to slow development and substantial hurdles. The absence of traditional transfer learning, which refers to the transfer of knowledge occurring only once at the beginning of the child model’s training, may result in the child model failing to fully assimilate the knowledge from the parent model. This could potentially lead to overfitting when translating low-resource languages. Our study employs a MNMT approach based on sustained transfer learning (STL) to address these issues in MNMT, whereby the child model acquires cross-model consistency knowledge from both synthetic parent data and child data. This entails simultaneously feeding each instance of the child data and its semantically equivalent counterpart of the synthetic parent data to both the child model and parent model. Subsequently, the objective is to align the two distributions obtained from the forward propagation, with the aim of facilitating STL. Finally, we demonstrate the effectiveness of our approach in improving the translation quality of MNMT through an ablation study and a case study. In comparison to other methods, our approach demonstrated the greatest improvement, with a maximum increase of 4.52 BLEU score.