AI vs. human: comparing learning experiences and performance through AI-generated and human-made instructional videos
摘要
We investigate how students’ learning experiences and performance differ when engaging with an AI-generated instructional video versus a human-made video. We address two new aspects when developing the AI-generated video: (1) Students’ connection with the character of the AI-video, the character being the instructor of the course, minimizing thus potential disruptions and (2) the technical aspect of the content. The study followed a quasi-experimental, non-randomized matched-group design involving 69 students who were enrolled in a business mathematics course offered during their first year of study and were taught by the same professor. The participants were assigned to two groups: the experimental group where students watched an AI-generated instructional video, and a control group where students watched a human-made instructional video. The learning experience was measured using five dimensions, namely comprehension, learner engagement, pace, instructor presence and overall satisfaction. The learning performance was measured using a retention test and a transfer test. We found no significant differences in the learning performance between the two groups. However, we found significant differences between the groups on learner engagement with small to medium effect size and on instructor presence with medium effect size. This suggests that AI-generated videos may achieve comparable learning outcomes to traditional videos in specific, controlled contexts. Finally, we underscore the potential of AI-generated videos as scalable educational tools while noting the need for further refinement in areas like voice quality and for future research involving larger, more diverse samples.