Multimodal Learning Analytics for Predicting Learning Gains in Online Service-Learning Programs
摘要
The growth of synchronous online education has introduced new forms of communication, collaboration, and engagement in real-time learning environments. To understand these processes, analytic approaches are needed that move beyond traditional assessments and make use of the multimodal signals available in virtual classrooms. This study investigates online service-learning programs as a representative setting where university students interact with peers and service recipients via videoconferencing. We construct a multimodal dataset of synchronous service-learning sessions, comprising transcripts, speech recordings, and screen recording content aligned at the utterance level. Using this dataset, we compare unimodal models with a range of multimodal fusion strategies to evaluate how linguistic, acoustic, and visual information contribute to predicting students’ self-reported learning gains. The results show that transcript features yielded higher predictive performance than other single modalities, while their integration with voice features and screen content produced more consistent and reliable predictions. These findings suggest that combining linguistic, acoustic, and visual information—capturing what students say, how they use their voices, and the learning materials they share on screen—offers a more grounded understanding of student learning processes in synchronous online education.