Designing and Evaluating Generative AI-Based Voice-Interaction Agents for Improving L2 Learners’ Oral Communication Competence

Oral communication competence is an essential skill in 21st century society and workplaces. However, despite years of English learning, non-native learners often struggle to communicate with others in English fluently and confidently. This study aims to explore the integration of language teaching theories and AI technologies to design and evaluate a Large Language Model (LLM)-enabled voice-based agent. Specifically, the design integrates Communicative Language Teaching (CLT) theories to instruct generative models. The evaluation adopts an experimental research design, wherein we evaluate the educational effectiveness of using the voice agent among 50 Chinese tertiary students. Pre- and post-assessments will be conducted to analyse learners’ oral proficiency development. To understand the mechanisms behind, a widely used coding scheme will be adapted to describe dialogue acts (DAs) at the speaking-turn level. Pattern mining algorithms will be employed to identify prominent DA patterns associated with students’ improvement in oral proficiency, shedding light on effective and ineffective interactional strategies employed by AI agents for further design considerations. This research aims to contribute both locally effective solutions for enhancing L2 learners’ oral communicative competence and generalisable design principles for AI applications in language teaching.