Fuzzy Logic-Based Metric for Machine Translation Evaluation

The globalized information-sharing phenomena facilitated by technologies such as the Internet have increased the demand for translation services. Automating translation has been at the forefront of solutions to address the demand. Automatic translation services have been available for sometimes provided by tech companies such as Google; however, achieving full translation accuracy is an ongoing challenge. In this paper, a fuzzy logic-based evaluation metric is proposed for evaluating machine translation accuracy. Evaluation results generated by the metric is compared with evaluation results generated by the bilingual evaluation understudy (BLEU) which is one of the most widely used machine translation accuracy evaluation metrics. The accuracy of evaluation results produced by both metrics are benchmarked against human-based translation accuracy evaluations for over a set of sentences translated from Turkish to English by tools Google translation, Yandex translation, and a simple neural machine translation prototype developed by the authors. The results show that the proposed fuzzy logic-based metric evaluates the accuracy of machine translations more effectively than the BLEU metric.