Evaluating Large Language Models for Diacritic Restoration in Romanian Texts: A Comparative Study
摘要
Automatic diacritic restoration is crucial for text processing in languages with rich diacritical marks, such as Romanian. This study evaluates the performance of various large language models (LLMs) in restoring diacritics in Romanian texts. Utilizing a comprehensive corpus, we tested models including OpenAI’s GPT-3.5, GPT-4, and GPT-4o, Google’s Gemini 1.0 Pro, Meta’s Llama 2 and 3, MistralAI’s Mixtral 8x7B Instruct, Deepinfra’s airoboros 70B, and OpenLLM-Ro’s RoLlama 2 7B, across different prompt templates ranging from zero-shot to complex multi-shot instructions. Our findings indicate that models such as OpenAI’s GPT-4o achieve high diacritic restoration accuracy, significantly surpassing a baseline echo model. However, other models, specifically those from Meta’s Llama family, showed varied performance, highlighting the impact of model architecture and training data on task-specific outcomes. This research underscores the need for specialized finetuning and model enhancements to improve NLP tasks involving diacritic-rich languages, providing valuable insights for future developments in computational linguistics.