Exploring Generative AI Models for Constructive Feedback in Programming Tasks
摘要
Providing timely and constructive feedback in large-scale programming courses remains a persistent educational challenge. While automated grading can assess correctness, it lacks the explanatory depth required to support student learning and improvement. This study explores the potential of Generative AI models specifically GPTo1, Claude 3.7, Cohere, and DeepSeekV3 in generating constructive feedback comparable to that of human instructors. We evaluated the outputs of these models based on linguistic features (e.g., readability, lexical diversity, sentence complexity) and semantic alignment with teacher feedback using metrics such as cosine similarity, STS scores, sentiment intensity, and named entity recognition. The results show that GPTo1 and Claude 3.7 most closely mirrored human feedback, achieving high scores in both readability and semantic alignment. GPTo1 achieved the highest STS score (0.83), cosine similarity (0.76), and readability score (63.89), while all models maintained a constructive tone with sentiment scores above 0.94. However, low NER overlap (e.g., 0.37 for GPTo1) revealed limitations in context-specific references. These findings highlight specific strengths of Generative AI models in replicating the linguistic structure and tone of human feedback, making them suitable candidates for integration into educational support tools.