Automated Topic Analysis with Large Language Models

Topic modeling is a popular method in tourism data analysis. Many authors have applied various approaches to summarize the main themes of travel blogs, reviews, video diaries, and similar media. One common shortcoming of these methods is their severe limitation in working with short documents, such as blog readers’ feedback (reactions). In the past few years, a new crop of large language models (LLMs), such as ChatGPT, has become available for researchers. We investigate LLM capability in extracting the main themes of viewers’ reactions to popular videos of a rural China destination that explores the cultural, technological, and natural heritage of the countryside. We compare the extracted topics and model accuracy with the results of the traditional Latent Dirichlet Allocation approach. Overall, LLM results are more accurate, specific, and better at separating discussion topics.