Automatic diacritic restoration is crucial for text processing in languages with rich diacritical marks, such as Romanian. This study evaluates the performance of various large language models (LLMs) in restoring diacritics in Romanian texts. Utilizing a comprehensive corpus, we tested models including OpenAI’s GPT-3.5, GPT-4, and GPT-4o, Google’s Gemini 1.0 Pro, Meta’s Llama 2 and 3, MistralAI’s Mixtral 8x7B Instruct, Deepinfra’s airoboros 70B, and OpenLLM-Ro’s RoLlama 2 7B, across different prompt templates ranging from zero-shot to complex multi-shot instructions. Our findings indicate that models such as OpenAI’s GPT-4o achieve high diacritic restoration accuracy, significantly surpassing a baseline echo model. However, other models, specifically those from Meta’s Llama family, showed varied performance, highlighting the impact of model architecture and training data on task-specific outcomes. This research underscores the need for specialized finetuning and model enhancements to improve NLP tasks involving diacritic-rich languages, providing valuable insights for future developments in computational linguistics.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating Large Language Models for Diacritic Restoration in Romanian Texts: A Comparative Study

  • Mihai Dan Nadăș,
  • Laura Dioșan

摘要

Automatic diacritic restoration is crucial for text processing in languages with rich diacritical marks, such as Romanian. This study evaluates the performance of various large language models (LLMs) in restoring diacritics in Romanian texts. Utilizing a comprehensive corpus, we tested models including OpenAI’s GPT-3.5, GPT-4, and GPT-4o, Google’s Gemini 1.0 Pro, Meta’s Llama 2 and 3, MistralAI’s Mixtral 8x7B Instruct, Deepinfra’s airoboros 70B, and OpenLLM-Ro’s RoLlama 2 7B, across different prompt templates ranging from zero-shot to complex multi-shot instructions. Our findings indicate that models such as OpenAI’s GPT-4o achieve high diacritic restoration accuracy, significantly surpassing a baseline echo model. However, other models, specifically those from Meta’s Llama family, showed varied performance, highlighting the impact of model architecture and training data on task-specific outcomes. This research underscores the need for specialized finetuning and model enhancements to improve NLP tasks involving diacritic-rich languages, providing valuable insights for future developments in computational linguistics.