GPT and Its Ability to Tell Stories—A Study

Creating engaging stories is an inherently human art, and the possibility of a computational model being able to mimic this task has long been a topic of discussion as it requires the unique intersection of the fields of artificial intelligence, psychology and literature. Computational generation of stories helps reduce efforts, find inspiration, and tailor stories to the user’s education or entertainment needs. This study compares the performance of the GPT-2 model when trained on the most popular datasets used in the task of story generation and evaluates the stories using both automatic and human evaluation, serving as a comparison of the datasets and of the evaluation metrics used as well. The GPT-2 model is used as the base model for the task and is trained independently on each dataset under evaluation to generate their respective scores and get a closer look at the performance of each dataset along with any drawbacks. Metrics like BLEU and ROGUE scores are used for automatic evaluation, with a more qualitative focus during human evaluation. The study concludes with insights on possible improvements to datasets and the GPT-2 model that can help improve story generation efforts.