ICBench: Benchmarking Knowledge Mastery in Introductory Computer Science Education

In computer science education, a fundamental challenge is to accurately assess a student’s knowledge mastery. Inspired by Knuth’s view that “the ultimate test of whether I understand something is if I can explain it to a compute”, we introduce the Knuth test and construct ICBench for benchmarking knowledge mastery in introductory computer science education. Three metrics of knowledge coverage, Bloom’s taxonomy, and traditional score are used in ICBench, where the former two assess the breadth and depth of a student’s mastery, respectively. We present the Rule-Property-Crux-Cohesion (RPCC) model for modeling a knowledge point and the Encode-Construct-Personalize (ECP) method to generate a personalized question set for each student. By analyzing the data of over 1000 students in the UCAS CS101 course from 2021 to 2023, we find that ICBench improves both knowledge coverage and Bloom’s taxonomy level. Furthermore, students who passed the test in ICBench outperformed their peers on final exams, scoring an average of 14% higher.