Evolving Standards of NLP Evaluation: A Survey of Metrics for Text Generation and Understanding
摘要
Within the domain of Natural Language Processing, a lot of work has been carried out for various tasks like question answering, text summarization, machine translation, image labeling, etc. To check the performance and effectiveness of the model, it is necessary to perform an evaluation of that model using quantitative measures. In this paper, we have discussed state-of-the-art automatic evaluation metrics like BLEU, METEOR, TER, ROUGE, CHRF and Neural Metrics like BERT Score, COMET and BLEURT. Precision, Recall, and F1 Score highlight the accuracy and completeness of model predictions. Text similarity is assessed through n-gram overlap, as measured by metrics such as ROUGE and BLEU. METEOR, CHRF, and TER integrate linguistic characteristics and edit distance measures to deliver more comprehensive and reliable evaluations. BERTScore, BLEURT, and COMET utilize pretrained language models to understand the underlying semantics and contextual relevance of text. These metrics ensure a higher degree of reliability in assessing machine translation and text generation quality.