Scholarlink

This study investigates the performance of modern multilingual Large Language Models (LLMs), including Qwen, LLaMA variants, Phi-4, Mistral, and DeepSeek on Nepali Question Answering (QA) tasks, a largely underexplored area in low-resource language processing. We employ both human and LLM-based evaluations, along with embedding-based semantic evaluation (using BERTScore for precision, recall, and F1-Score), based on four key criteria: correctness and relevance, language quality, context understanding, and conciseness and informativeness. The experiment utilized a Nepali QA dataset that was developed to check the model’s capability to accommodate the linguistic nuances. The result revealed that Phi-4 outperformed all other models, therefore illustrating that it has an accurate and context-aware interpretation of Nepali semantics. The research highlights the advantages and limitations of multilingual LLMs in low-resource conditions and it also assists in selecting the appropriate model and arranging future studies in Nepali NLP.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Comparative Evaluation of Modern Large Language Models for Nepali Language Generation

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Niranjan Shrestha,
  • Sharad K. Ghimire

摘要

This study investigates the performance of modern multilingual Large Language Models (LLMs), including Qwen, LLaMA variants, Phi-4, Mistral, and DeepSeek on Nepali Question Answering (QA) tasks, a largely underexplored area in low-resource language processing. We employ both human and LLM-based evaluations, along with embedding-based semantic evaluation (using BERTScore for precision, recall, and F1-Score), based on four key criteria: correctness and relevance, language quality, context understanding, and conciseness and informativeness. The experiment utilized a Nepali QA dataset that was developed to check the model’s capability to accommodate the linguistic nuances. The result revealed that Phi-4 outperformed all other models, therefore illustrating that it has an accurate and context-aware interpretation of Nepali semantics. The research highlights the advantages and limitations of multilingual LLMs in low-resource conditions and it also assists in selecting the appropriate model and arranging future studies in Nepali NLP.