Within the domain of Natural Language Processing, a lot of work has been carried out for various tasks like question answering, text summarization, machine translation, image labeling, etc. To check the performance and effectiveness of the model, it is necessary to perform an evaluation of that model using quantitative measures. In this paper, we have discussed state-of-the-art automatic evaluation metrics like BLEU, METEOR, TER, ROUGE, CHRF and Neural Metrics like BERT Score, COMET and BLEURT. Precision, Recall, and F1 Score highlight the accuracy and completeness of model predictions. Text similarity is assessed through n-gram overlap, as measured by metrics such as ROUGE and BLEU. METEOR, CHRF, and TER integrate linguistic characteristics and edit distance measures to deliver more comprehensive and reliable evaluations. BERTScore, BLEURT, and COMET utilize pretrained language models to understand the underlying semantics and contextual relevance of text. These metrics ensure a higher degree of reliability in assessing machine translation and text generation quality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evolving Standards of NLP Evaluation: A Survey of Metrics for Text Generation and Understanding

  • S. S. Barphe,
  • S. R. Sutar,
  • A. W. Kiwelekar,
  • S. M. Jadhav,
  • V. J. Kadam

摘要

Within the domain of Natural Language Processing, a lot of work has been carried out for various tasks like question answering, text summarization, machine translation, image labeling, etc. To check the performance and effectiveness of the model, it is necessary to perform an evaluation of that model using quantitative measures. In this paper, we have discussed state-of-the-art automatic evaluation metrics like BLEU, METEOR, TER, ROUGE, CHRF and Neural Metrics like BERT Score, COMET and BLEURT. Precision, Recall, and F1 Score highlight the accuracy and completeness of model predictions. Text similarity is assessed through n-gram overlap, as measured by metrics such as ROUGE and BLEU. METEOR, CHRF, and TER integrate linguistic characteristics and edit distance measures to deliver more comprehensive and reliable evaluations. BERTScore, BLEURT, and COMET utilize pretrained language models to understand the underlying semantics and contextual relevance of text. These metrics ensure a higher degree of reliability in assessing machine translation and text generation quality.