Machine Learning Models to Detect AI-Assisted Code Anomaly in Introductory Programming Course
摘要
The use of AI tools to complete class assignments has become a trend in recent years. The instructors’ tasks become more complex in monitoring and evaluating students’ progress. In this chapter, we experimented with various machine learning approaches for automatic anomaly detection, such as tree-based and parametric-based models. We also deepen the models with a neural-based approach and feature selection method. The datasets are derived from student weekly submissions from a Python introductory programming course in the first semester of the 2023–2024 academic year. The random committees model shows the best performance using all features with an accuracy of 96.7% and a recall rate of 62.9%. Our observation indicates special features that characterize each assignment type: branching, looping, and functions. Further experiments show that the accuracy of our final meta-learning models is above 95% during the testing by using the real dataset from students’ submissions, with the best achieved for the branching assignment type by using all features. Based on these performances, we believe that our feature sets and meta-learning approach would be promising to detect anomalies in AI-assisted code submissions.