AI-Generated Code Detection: An Examination of Current Tools in Education
摘要
The increasing availability of large language models (LLMs) has raised significant concerns over academic integrity in introductory programming courses. Existing AI detectors, primarily developed for natural language, have shown limited efficacy when identifying AI-generated code, particularly after simple obfuscations. This study evaluates seven state-of-the-art detection tools on Python solutions collected from an educational platform (Senecode). We built a new dataset of 822 human-written and 822 AI-generated code samples, with the AI samples systematically modified using six prompt variants. Our findings reveal critical precision-recall tradeoffs and notable performance drops in the face of minor obfuscation. Our findings reveal that current AI-detection tools are unreliable for educational use and highlight the need to shift focus toward rethinking assessment practices in the age of generative AI.