Human and Machine Complementary Roles in Collaborative Evaluation of Creative Speech

This paper explores the complementary roles of humans and machines in the collaborative evaluation of creative speech, aiming to enhance the accuracy, reliability, and efficiency of creativity assessments. Traditional methods of evaluating creative speech primarily rely on human judges, who bring intuition and contextual understanding but are limited by subjectivity and scalability challenges. Conversely, artificial intelligence offers computational power and consistency but lacks the complex judgment that human evaluators provide. This research aims to investigate the divergent and convergent aspects of each evaluation and highlight the importance of co-creative collaboration between humans and machines. We utilize the Torrance Tests of Creative Thinking to assess the 22 creative speech samples of 11 participants, comparing evaluations conducted by human experts, and an AI system (the GPT4). While the results demonstrate a certain degree of correlation between human and AI evaluation especially in evaluating Flexibility, they also reveal the differences in how humans and machines perceive Originality in text.