ActiveAI: The Effectiveness of an Interactive Tutoring System in Developing K-12 AI Literacy

As we witness groundbreaking advancements in Artificial Intelligence (AI), it is clear that the next generation must be equipped with AI literacy: the skill to interact, evaluate, and collaborate with AI systems. This study introduces ActiveAI, a scalable web-based tutoring system aligned with AI4K12’s five big ideas in AI, designed to foster AI literacy among K-12 students through active learning and interaction with intelligent agents. A controlled classroom study involving 171 middle school learners was conducted to assess the effectiveness of ActiveAI in fostering AI literacy skills and competency toward AI. Results showed that, compared to students in the tell-and-practice control condition, students who used ActiveAI exhibited higher post-test performance in the module about how next-word prediction and temperature work in large language models. Students also developed higher self-reported competence toward AI after using ActiveAI than in the control condition. We conclude by suggesting assessment designs that promote deeper engagement with AI concepts by addressing students’ common misconceptions, like “AI thinks just like humans”, in K-12 AI literacy education.