Automatic Reference-Free Fine-Grained Machine Translation Error Detection via Named Entity Recognition and Back-Translation

Prior researches in word-level machine translation quality estimation (QE) have made significant strides in detecting superfluous and omitted translations. Nevertheless, these approaches rely heavily on extensive reference data and struggle to effectively differentiate between superfluous translations, missing translations and mistranslations, resulting in lower detection probabilities. To address this limitation, we propose an Automatic Reference-Free Fine-Grained Neural Machine Translation Error Detection method (ARFGED) that leverages Named Entity Recognition and Back-Translation. A Named Entity Recognition (NER) tool is utilized to get initial error types probability related to entity translation. Back-translation inference is applied to the multilingual machine translation model to obtain fine-grained error types, achieving automatic and reference-free translation error detection. Subsequently, the combination of two error types above are used to train a classifier for clearer distinction between superfluous translations, omissions and incorrect translations. Experimental results on original dataset and our synthetic dataset demonstrate that the proposed method achieves significant improvements in F1 scores compared to supervised and contrastive conditioning methods.