Assessing the Effectiveness of AI Detection Tools for Varying Levels of AI-Generated Text
摘要
Generative AI models are now so advanced as to blur the lines between human and machine-generated texts, and as a rule of thumb, it becomes harder each day. This work is aimed at the evaluation of the capability of the following AI detection tools to detect AI-generated texts: Copyleaks, GPTZero, and ZeroGPT. To this end, we utilize a dataset of 1000 documents that have been divided into four groups: 100% AI-generated, 75% AI-generated, 50% AI-generated, and 25% AI-generated text. The tools are hence judged by means of some quality prediction metrics in terms of their effectiveness in detecting AI-generated content. It is important to note that Copyleaks, along with other AI detection tools, is currently widely used in the market to detect both human-written and machine-generated content. This highlights the necessity of Copyleaks in copy detection as it the most effective tool for AI content detection. It can be noticed that ZeroGPT is fragile and fails at 100% recall, and although GPTZero is as strong in recall as it is weak in slower processing speed, it lacks in precision. The elaboration clarifies the advantages and shortcomings of each tool, emphasizing the difficulty that still exists in AI text detection, especially concerning the texts, which may include both human-written and AI-generated parts. These results are not only of great value to researchers and educators, but also highly demanded by content verification platforms that are searching for AI detection solutions which are very efficient. Besides, the paper points to the necessity of the further development of AI detection technologies to cope with the rising level of AI-generated texts. More importantly, future research should focus on the advances of generative models and their impact on human-AI cooperation, and the integrating of detection tools into prolong verification systems.