An Approach to Mongolian Neural Machine Translation Based on RWKV Language Model and Contrastive Learning

Low-resource machine translation (LMT) is a challenging task, especially for languages with limited resources like Mongolian. In this paper, we propose a novel Mongolian-to-Chinese machine translation approach based on the RWKV language model and augmented with contrastive learning. Traditional methods that perturb the embedding layer often suffer from issues such as semantic distortion and excessive perturbation, leading to training instability. To address these problems, we introduce a contrastive learning approach combined with adversarial perturbation. Additionally, the RWKV language model, as a new architecture, has shown to be more efficient in terms of training and inference time compared to traditional transformer models in various natural language processing tasks. In this work, we employ the RWKV language model as the core of our machine translation model. We evaluate our approach on a benchmark dataset of Mongolian-to-Chinese parallel sentences. The experimental results demonstrate that our method outperforms the state-of-the-art approaches in Mongolian machine translation. Furthermore, our research indicates that the proposed approach significantly mitigates the training instability caused by adversarial perturbation and demonstrates the effectiveness of employing the RWKV language model in improving translation performance.