Pre-trained Models for Grammatical Error Correction in Healthcare-Specific Text
摘要
Grammatical Error Correction (GEC) is a common Natural Language Processing task and has been studied extensively in the field. There is an added complexity when trying to correct domain-specific text that is uncommon in general-purpose corpora. In this paper, we present a comparative study of three different approaches using pre-trained language models for Grammatical Error Correction in healthcare-specific text. We evaluated the performance of all proposals using the GLEU score metric, which allows a quantitative comparison of all methods. We utilized two different problem settings: first, a sequence-to-sequence pre-trained T5 model, followed by a fine-tuning process over a small set of examples. The second model is a pre-trained large language model, for which we test both zero-shot and few-shot in-context learning. The study shows how the fine-tuned T5 model is capable of exceeding the performance shown by the LLM tested. With this result, we show how smaller encoder-decoder models can solve domain-specific tasks with fewer parameters than a purely generative pre-trained LLM.