Assessing Computational Thinking in Artificial Intelligence Training Context: Design Principles, Content Validation, and Pilot Testing
摘要
Computational thinking (CT) is an essential skill in the twenty-first century. The rapid rise of Artificial Intelligence (AI) has changed the way CT is educated. Although abundant resources for teaching and assessing CT have been developed, research on integrating AI knowledge into CT remains scarce. Additionally, current CT assessments mainly target K–12 students and rely on block-based programming frameworks, leaving a gap for higher education where text-based programming would be more appropriate. To address these gaps, we developed the Computational Thinking in AI Training Test (CTAT), tailored for college and university students in CS- and AI-related programs, with Python serving as the primary programming language. We employed a principled approach, Evidence-Centered Design (ECD) to develop the test, and a total of 34 items were created. The test was then validated via expert review and cognitive interviews with students, and revisions were made based on the feedback. A pilot study was then conducted at a public college in southern China, and 139 students with a CS/AI-related background agreed to participate. Item analysis was performed, and the results indicate that CTAT is suitable for the target students. Suggestions for further refinement were proposed, and future research directions are discussed.