Complementary advantages exhibited by ChatGPT and human readers in text reading inference
摘要
This study aims to investigate how ChatGPTs and high school students exhibit inferential ability in English narrative reading and to compare how the two ChatGPTs perform when prompts are updated elaborately. The participants were 114 Chinese senior school students, GPT-3.5, and GPT-4.0. The whole study consisted of three tests: Test 1 for commonsense inference, Test 2 for emotional inference, and Test 3 for causal inference. ChatGPTs were required to generate responses to each test according to the same prompts that were provided for the students. The results revealed that the students outperformed the two ChatGPTs in local-culture-related inferences but performed worse in daily life inferences. With respect to emotional inference, GPT-4.0 excelled whereas GPT-3.5 lagged in terms of judgment accuracy. Additionally, the students demonstrated better logical analysis as compared to both chatbots. In the updating prompt condition, GPT-4.0 displayed enhanced causal reasoning ability, approximately at the level of human readers, whereas GPT-3.5 remained unchanged. This study discloses that human readers and the two versions of ChatGPTs have their respective dis/advantages in drawing inferences from reading comprehension, consequently complementing each other in text cognition. In language teaching and learning, both teachers and students can leverage ChatGPTs’ unique inferential ability to improve students’ English reading proficiency.