Optimizing the Accuracy of Chatbot Applications Based on the GPT-3.5 Turbo Platform of OpenAI to Provide Service Prices to Customers

Artificial Intelligence (AI) in general and chatbots in particular have emerged as significant investments in the healthcare sector in Vietnam, notably at Hanoi Medical University Hospital. The integration of virtual assistants to provide information for patients at Hanoi Medical University Hospital can alleviate the burden on human resources and facilitate the hospital’s transition toward a futuristic healthcare facility. This study employs Large Language Models, specifically the gpt-3.5-turbo model from OpenAI, to construct a chatbot capable of answering user queries about the prices of various services offered at the hospital. The chatbot undergoes fine-tuning using a dataset comprising predefined questions and answers from the hospital. The study also utilizes similarity metrics such as Jaccard similarity to evaluate the accuracy of the chatbot. After three rounds of fine-tuning, with each dataset subjected to data augmentation and three epochs run per fine-tuning iteration, the accuracy of the chatbot has improved significantly from 63% to 82%, culminating in an impressive 96%. However, being a data-hungry model, the development of the chatbot encounters challenges due to the limited dataset available at the hospital. This study highlights the substantial improvement in the chatbot’s performance through iterative fine-tuning and data augmentation. Nonetheless, evaluating the model’s accuracy remains challenging with the current limited dataset. Future developments may involve incorporating additional evaluation metrics, such as direct user feedback from chatlogs, to enhance the assessment of the chatbot and further refine it for user deployment. The study provides insights into the promising potential of AI-driven chatbots in transforming healthcare information delivery, emphasizing the importance of ongoing enhancements and user-centric evaluations in optimizing their functionality.