An Improvement of Large Language Model for Vietnamese Traffic Law Question Answer System

Generative models have recently started to outperform extractive models in Question-Answering (QA) tasks with the rise of Large Language Models (LLMs). In this paper, we propose a QA System for Vietnamese traffic law that utilizes LLMs and also combined it with a Retrieval Module to perform better results. The LLM we used in the system is Vietcuna, which has been pre-trained from BloomZ. For the retrieval task, we fine-tuned PhoBERT following the approach of SimCSE, a simple contrastive learning framework for our data. We also conducted the LLM Judge approach, which uses GPT-4 to evaluate the QA task. The experimental results showed that our system works well in answering questions about Vietnamese traffic law. Furthermore, this method holds potential for application across various fields involving the Vietnamese language.