Teachers often struggle to assess oral presentations in real time due to multitasking and cognitive load. During live assessments, the teacher must simultaneously listen to the student’s discourse, observe non-verbal communication, analyze slide content, and take structured notes. This multitasking often leads to incomplete or imprecise evaluations, especially regarding the verbal, paraverbal, and non-verbal dimensions. This paper introduces EvalIA, a hybrid evaluation method that combines artificial intelligence (AI) and human judgment. The aim is to assist teachers in analyzing specific oral performance criteria while offering students detailed and reusable feedback. To explore its feasibility, five student presentations were recorded, transcribed, and analyzed using ChatGPT, using three criteria: originality, verbal communication, and paraverbal features. The AI-generated scores were then compared to the teacher’s assessments using Cohen’s Kappa coefficient to evaluate the level of agreement. The results show strong convergence on technical criteria such as rhythm and lexical structure, while discrepancies appeared in more subjective aspects like fluency. One case highlighted the limits of automatic transcription when articulation is unclear. EvalIA demonstrates potential as a supportive assessment tool: it helps relieve the teacher’s cognitive load, increases the fairness of evaluations, and empowers students to understand better and improve their oral performance. Future work will include testing the method on a larger sample to further validate its reliability and scalability in educational settings and ultimately support the creation of an automated tool for oral performance analysis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evalia: Artificial Intelligence and Human Judgment for the Evaluation of Oral Presentations

  • Amine Marouki

摘要

Teachers often struggle to assess oral presentations in real time due to multitasking and cognitive load. During live assessments, the teacher must simultaneously listen to the student’s discourse, observe non-verbal communication, analyze slide content, and take structured notes. This multitasking often leads to incomplete or imprecise evaluations, especially regarding the verbal, paraverbal, and non-verbal dimensions. This paper introduces EvalIA, a hybrid evaluation method that combines artificial intelligence (AI) and human judgment. The aim is to assist teachers in analyzing specific oral performance criteria while offering students detailed and reusable feedback. To explore its feasibility, five student presentations were recorded, transcribed, and analyzed using ChatGPT, using three criteria: originality, verbal communication, and paraverbal features. The AI-generated scores were then compared to the teacher’s assessments using Cohen’s Kappa coefficient to evaluate the level of agreement. The results show strong convergence on technical criteria such as rhythm and lexical structure, while discrepancies appeared in more subjective aspects like fluency. One case highlighted the limits of automatic transcription when articulation is unclear. EvalIA demonstrates potential as a supportive assessment tool: it helps relieve the teacher’s cognitive load, increases the fairness of evaluations, and empowers students to understand better and improve their oral performance. Future work will include testing the method on a larger sample to further validate its reliability and scalability in educational settings and ultimately support the creation of an automated tool for oral performance analysis.