SimCLNMT: A Simple Contrastive Learning Method for Enhancing Neural Machine Translation Quality
摘要
Neural Machine Translation (NMT) models are typically trained using Maximum Likelihood Estimation (MLE). However, this approach has a limitation: while it might select the best word for the immediate context, it does not generally optimize for the entire sentence. To mitigate this issue, we propose a simple yet effective training method called SimCLNMT. This method is designed to select words that fit well in the immediate context and also enhance the overall translation quality over time. During training, SimCLNMT scores multiple system-generated (candidate) translations using the logarithm of conditional probabilities. It then employs a ranking loss function to learn and adjust these probabilities to align with the corresponding quality scores. Our experimental results demonstrate that SimCLNMT consistently outperforms traditional MLE training on both the NIST English-Chinese and WMT’14 English-German datasets. Further analysis also indicates that the translations generated by our model are more closely aligned with the corresponding quality scores. We release our code at https://github.com/chaos130/fairseq_SimCLNMT .