The use of AI tools to complete class assignments has become a trend in recent years. The instructors’ tasks become more complex in monitoring and evaluating students’ progress. In this chapter, we experimented with various machine learning approaches for automatic anomaly detection, such as tree-based and parametric-based models. We also deepen the models with a neural-based approach and feature selection method. The datasets are derived from student weekly submissions from a Python introductory programming course in the first semester of the 2023–2024 academic year. The random committees model shows the best performance using all features with an accuracy of 96.7% and a recall rate of 62.9%. Our observation indicates special features that characterize each assignment type: branching, looping, and functions. Further experiments show that the accuracy of our final meta-learning models is above 95% during the testing by using the real dataset from students’ submissions, with the best achieved for the branching assignment type by using all features. Based on these performances, we believe that our feature sets and meta-learning approach would be promising to detect anomalies in AI-assisted code submissions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Machine Learning Models to Detect AI-Assisted Code Anomaly in Introductory Programming Course

  • Hapnes Toba,
  • Oscar Karnalim

摘要

The use of AI tools to complete class assignments has become a trend in recent years. The instructors’ tasks become more complex in monitoring and evaluating students’ progress. In this chapter, we experimented with various machine learning approaches for automatic anomaly detection, such as tree-based and parametric-based models. We also deepen the models with a neural-based approach and feature selection method. The datasets are derived from student weekly submissions from a Python introductory programming course in the first semester of the 2023–2024 academic year. The random committees model shows the best performance using all features with an accuracy of 96.7% and a recall rate of 62.9%. Our observation indicates special features that characterize each assignment type: branching, looping, and functions. Further experiments show that the accuracy of our final meta-learning models is above 95% during the testing by using the real dataset from students’ submissions, with the best achieved for the branching assignment type by using all features. Based on these performances, we believe that our feature sets and meta-learning approach would be promising to detect anomalies in AI-assisted code submissions.