We propose generating answers from the Japanese language section for large-scale official trial tests in Japanese university entrance examinations using Retrieval-Augmented Generation (RAG). RAG follows a process in which text segments with high similarity to a given query are retrieved from a database and incorporated into the prompt. Using the question as the query and appending similar segments from the main text to the prompt, we expect to improve the answer generation performance. To investigate this, we used large-scale official trial tests from Japanese university entrance examinations as a benchmark and generated answers using several large language models (LLMs) with and without RAG. The generated answers are then evaluated on the basis of the official scoring conditions. Experiments conducted with gemma2, llaMA3, and gpt-4o show that RAG is effective for relatively small and medium sized LLMs. However, for large-scale LLMs, using the model alone without RAG results in better answer generation. We further propose an enhanced version of RAG, named Re-retrieval RAG, which incorporates a recursive retrieval process to improve the accuracy and consistency of the generated answers. Our experimental results show that the proposed Re-retrieval RAG consistently achieved the highest scores across all six questions tested, demonstrating its general effectiveness.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Answer Generation for Large-Scale Official Trial Tests in Japanese University Entrance Exams Using RAG

  • Taketsuna Ichiyanagi,
  • Kanako Komiya,
  • Tsunenori Ishioka,
  • Masaki Nakagawa

摘要

We propose generating answers from the Japanese language section for large-scale official trial tests in Japanese university entrance examinations using Retrieval-Augmented Generation (RAG). RAG follows a process in which text segments with high similarity to a given query are retrieved from a database and incorporated into the prompt. Using the question as the query and appending similar segments from the main text to the prompt, we expect to improve the answer generation performance. To investigate this, we used large-scale official trial tests from Japanese university entrance examinations as a benchmark and generated answers using several large language models (LLMs) with and without RAG. The generated answers are then evaluated on the basis of the official scoring conditions. Experiments conducted with gemma2, llaMA3, and gpt-4o show that RAG is effective for relatively small and medium sized LLMs. However, for large-scale LLMs, using the model alone without RAG results in better answer generation. We further propose an enhanced version of RAG, named Re-retrieval RAG, which incorporates a recursive retrieval process to improve the accuracy and consistency of the generated answers. Our experimental results show that the proposed Re-retrieval RAG consistently achieved the highest scores across all six questions tested, demonstrating its general effectiveness.