This paper investigates the impact of real-time risk of failure predictions, accompanied by LIME-based textual explanations, on learner engagement and course achievement in an online professional learning context. The study involves 240 learners enrolled in 24 courses, divided into three groups: a control group receiving no predictions, a prediction-only group, and a treatment group receiving both risk predictions and textual explanations via a learning analytics dashboard. Engagement is measured using four behavioral indicators derived from LMS digital traces, calculated both before and after predictions. The results reveal that the prediction-only group exhibits an engagement increase of approximately 12–15%, while the treatment group sees a further improvement of 18–22% compared to the control group. Notably, 69% of learners in the treatment group rate the predictions and textual explanations as useful. This group also records higher final quiz scores and course completion rates, indicating enhanced course achievement. Mixed-design ANOVA confirms the statistical significance of these findings, underscoring the importance of interpretable and transparent real-time predictions in supporting learner engagement and course achievement.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Transparent Risk Predictions and Explanatory Feedback: Boosting Engagement and Course Achievement in Online Professional Learning

  • Mohamed Mouaici

摘要

This paper investigates the impact of real-time risk of failure predictions, accompanied by LIME-based textual explanations, on learner engagement and course achievement in an online professional learning context. The study involves 240 learners enrolled in 24 courses, divided into three groups: a control group receiving no predictions, a prediction-only group, and a treatment group receiving both risk predictions and textual explanations via a learning analytics dashboard. Engagement is measured using four behavioral indicators derived from LMS digital traces, calculated both before and after predictions. The results reveal that the prediction-only group exhibits an engagement increase of approximately 12–15%, while the treatment group sees a further improvement of 18–22% compared to the control group. Notably, 69% of learners in the treatment group rate the predictions and textual explanations as useful. This group also records higher final quiz scores and course completion rates, indicating enhanced course achievement. Mixed-design ANOVA confirms the statistical significance of these findings, underscoring the importance of interpretable and transparent real-time predictions in supporting learner engagement and course achievement.