Recent advances in natural language processing have showcased the power of Large Language Models (LLMs), but evaluating their performance on Vietnamese multi-hop reasoning tasks remains challenging. Traditional approaches to creating Vietnamese evaluation datasets have been constrained by their reliance on manual processes and proprietary models, resulting in limited scalability and substantial resource requirements. We present a novel framework that automates the generation of Vietnamese multi-hop question-answering datasets by leveraging open-source models with enhanced prompting strategies. Our experiments reveal that despite the general advantages of proprietary models, targeted prompt optimization and linguistic adjustments enable open-source alternatives to deliver strong performance. The effectiveness of our approach shows that successful multilingual systems require both robust models and careful attention to language-specific requirements. By offering an efficient and scalable methodology for Vietnamese dataset development, our work helps democratize access to sophisticated language technology while ensuring output quality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automated Question-Answer Generation for Evaluating Vietnamese RAG-Based Approaches

  • Dinh-Huan Nguyen,
  • Long Nguyen

摘要

Recent advances in natural language processing have showcased the power of Large Language Models (LLMs), but evaluating their performance on Vietnamese multi-hop reasoning tasks remains challenging. Traditional approaches to creating Vietnamese evaluation datasets have been constrained by their reliance on manual processes and proprietary models, resulting in limited scalability and substantial resource requirements. We present a novel framework that automates the generation of Vietnamese multi-hop question-answering datasets by leveraging open-source models with enhanced prompting strategies. Our experiments reveal that despite the general advantages of proprietary models, targeted prompt optimization and linguistic adjustments enable open-source alternatives to deliver strong performance. The effectiveness of our approach shows that successful multilingual systems require both robust models and careful attention to language-specific requirements. By offering an efficient and scalable methodology for Vietnamese dataset development, our work helps democratize access to sophisticated language technology while ensuring output quality.