Scholarlink

This study investigated the efficacy of Large Language Models (LLMs) for providing formative assessment of spoken English within a mixed-methods, one-group pre-test/post-test design. The research involved 30 intermediate English as a Second Language (ESL) learners and five expert faculty members. The design integrated quantitative pre-test/post-test data with qualitative analysis of AI-generated feedback, student reflections, and faculty interviews. During the intervention, all students used LLM tools (ChatGPT and Google Gemini) for practice and feedback. To ensure a robust, triangulated assessment, a coder-based framework was implemented where human experts evaluated spoken tasks against linguistic criteria. A paired-samples t-test revealed a statistically significant improvement in speaking proficiency from pre-test to post-test. Thematic analysis of qualitative data indicated that LLM-generated feedback was perceived as useful, accessible, and effective in reducing learner anxiety. This multi-faceted approach affirms that LLMs can be effective supplementary tools in language education and offers a holistic model for their integration into formative assessment practices.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Formative Assessment of Spoken English Using Large Language Models in a Controlled Intervention Study

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Mahimaben Panjabi,
  • Vijay Makwana

摘要

This study investigated the efficacy of Large Language Models (LLMs) for providing formative assessment of spoken English within a mixed-methods, one-group pre-test/post-test design. The research involved 30 intermediate English as a Second Language (ESL) learners and five expert faculty members. The design integrated quantitative pre-test/post-test data with qualitative analysis of AI-generated feedback, student reflections, and faculty interviews. During the intervention, all students used LLM tools (ChatGPT and Google Gemini) for practice and feedback. To ensure a robust, triangulated assessment, a coder-based framework was implemented where human experts evaluated spoken tasks against linguistic criteria. A paired-samples t-test revealed a statistically significant improvement in speaking proficiency from pre-test to post-test. Thematic analysis of qualitative data indicated that LLM-generated feedback was perceived as useful, accessible, and effective in reducing learner anxiety. This multi-faceted approach affirms that LLMs can be effective supplementary tools in language education and offers a holistic model for their integration into formative assessment practices.