Efficacy of Selected Generative AI Systems in Assessing Discourse Coherence, Pragmatic Competence, Collocational Competence, and Figurative Language
摘要
Artificial Intelligence (AI) is rapidly transforming language education, functioning not merely as an automation tool but as a catalyst for understanding, accuracy, and individualized learning. Nonetheless, despite the extensive integration of AI in education, comprehension of its efficacy in assessing language proficiency, particularly these four parameters, discourse coherence, pragmatic competence, collocational competence, and figurative language, remains largely underexplored from the perspective of integrating AI. Consequently, capacity to embed AI driven formative assessments for enhanced feedback can be insightful. The present study assesses the efficacy of five leading generative AI systems namely GPT-4, Gemini, DeepSeek, Copilot, and Claude Sonnet 4, through standardised dialogues that encompass four linguistic criteria, discourse coherence, pragmatic competence, collocational competence, and figurative language. language. The dialogues constructed by students as a part of a classroom activity are assessed through each AI tool will through CEFR based 5-point rubrics. The study of this evaluation, aims to explore how effectively and precisely these AI tools process the specific language inputs deliver the feedback that aids learners in improving expression, increasing their clarity, and enriching their contextual comprehension. The findings of this study will benefit teachers, curriculum developers and language learners by offering evidence-based insights into which generative AI systems can be utilised for the improvement, accuracy and learner autonomy in higher education and EFL domains.