Scholarlink

This study investigates the enhancement of Romanized Nepali text generation and comprehension using open-source large language models (LLMs), addressing challenges posed by colloquial words and non-standard sentence structures. While current open-source LLMs struggle with these complexities, commercial models like OpenAI GPT-4 outperform in generating accurate Nepali text. To bridge this gap, we use OpenAI GPT-4 for synthetic data generation, manually verified for accuracy by a native speaker, and fine-tune an open-source LLM using the Parameter-Efficient Fine-Tuning (PEFT) technique. Our evaluation, based on translation quality metrics, shows a marked improvement in the fine-tuned model’s performance over the base model, demonstrating the effectiveness of this approach in low-resource language NLP. Additionally, this work contributes to the community by open-sourcing the fine-tuning process and the generated dataset.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing Nepali Text Understanding with Machine Translation and LoRA Fine-Tuning of Open-Source LLM

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Kshitiz Rimal,
  • Noorhan Abbas

摘要

This study investigates the enhancement of Romanized Nepali text generation and comprehension using open-source large language models (LLMs), addressing challenges posed by colloquial words and non-standard sentence structures. While current open-source LLMs struggle with these complexities, commercial models like OpenAI GPT-4 outperform in generating accurate Nepali text. To bridge this gap, we use OpenAI GPT-4 for synthetic data generation, manually verified for accuracy by a native speaker, and fine-tune an open-source LLM using the Parameter-Efficient Fine-Tuning (PEFT) technique. Our evaluation, based on translation quality metrics, shows a marked improvement in the fine-tuned model’s performance over the base model, demonstrating the effectiveness of this approach in low-resource language NLP. Additionally, this work contributes to the community by open-sourcing the fine-tuning process and the generated dataset.