In this paper we have shown the process of evaluation of MT systems without corelating the automatic evaluation with human evaluation. For this we have used statistical significance testing where we have used automatic evaluation metrics applied statistical testing. The results of the study were then given to humans for evaluation and it was found that this approach can be an alternative to corelating the automatic evaluation with human evaluation. We tested the results on 27 MT engines across six language pairs and found that the results of significance tests were at par the human correlations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating Machine Translation Outputs by Comparing Means

  • Abhimanyu Singh Kulhari,
  • Nisheeth Joshi

摘要

In this paper we have shown the process of evaluation of MT systems without corelating the automatic evaluation with human evaluation. For this we have used statistical significance testing where we have used automatic evaluation metrics applied statistical testing. The results of the study were then given to humans for evaluation and it was found that this approach can be an alternative to corelating the automatic evaluation with human evaluation. We tested the results on 27 MT engines across six language pairs and found that the results of significance tests were at par the human correlations.