Neural Machine Translation (NMT) models are typically trained using Maximum Likelihood Estimation (MLE). However, this approach has a limitation: while it might select the best word for the immediate context, it does not generally optimize for the entire sentence. To mitigate this issue, we propose a simple yet effective training method called SimCLNMT. This method is designed to select words that fit well in the immediate context and also enhance the overall translation quality over time. During training, SimCLNMT scores multiple system-generated (candidate) translations using the logarithm of conditional probabilities. It then employs a ranking loss function to learn and adjust these probabilities to align with the corresponding quality scores. Our experimental results demonstrate that SimCLNMT consistently outperforms traditional MLE training on both the NIST English-Chinese and WMT’14 English-German datasets. Further analysis also indicates that the translations generated by our model are more closely aligned with the corresponding quality scores. We release our code at https://github.com/chaos130/fairseq_SimCLNMT .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SimCLNMT: A Simple Contrastive Learning Method for Enhancing Neural Machine Translation Quality

  • Menglong Xu,
  • Yanliang Zhang

摘要

Neural Machine Translation (NMT) models are typically trained using Maximum Likelihood Estimation (MLE). However, this approach has a limitation: while it might select the best word for the immediate context, it does not generally optimize for the entire sentence. To mitigate this issue, we propose a simple yet effective training method called SimCLNMT. This method is designed to select words that fit well in the immediate context and also enhance the overall translation quality over time. During training, SimCLNMT scores multiple system-generated (candidate) translations using the logarithm of conditional probabilities. It then employs a ranking loss function to learn and adjust these probabilities to align with the corresponding quality scores. Our experimental results demonstrate that SimCLNMT consistently outperforms traditional MLE training on both the NIST English-Chinese and WMT’14 English-German datasets. Further analysis also indicates that the translations generated by our model are more closely aligned with the corresponding quality scores. We release our code at https://github.com/chaos130/fairseq_SimCLNMT .