The widespread use of AI-generated text introduces ethical concerns surrounding plagiarism, bias, and authenticity. Traditional detection systems, such as classifier-based approaches and linguistic feature extraction methods, have been developed to identify AI-generated content in domains like code, images, and text. However, these methods often suffer from limited generalizability across domains and decreasing accuracy as AI models become more fluent and human-like. Most current approaches evaluate AI output along a narrow set of dimensions, typically focusing on surface-level features such as fluency, syntax, or token frequency. This paper proposes a structured and multidimensional framework to evaluate AI-generated versus human-authored text across six key linguistic dimensions: grammatical consistency, vocabulary diversity, emotional expression, personalization, sensitivity to controversial topics, and response consistency. Using the HC3 dataset of 24,000 paired responses, we analyzed the differences between human and AI-generated text. Results show that human responses exhibit greater emotional diversity, vocabulary richness, and personalization, while AI outperforms grammatical accuracy and consistency. These findings contribute to a more holistic assessment of AI language models and highlight the need for evaluation methods that extend beyond traditional binary classification or syntax-based scoring.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Beyond Syntax: Evaluating the Depth, Bias, and Expressiveness of Human vs. AI-Generated Text

  • Raja Shaker Chinthakindi,
  • Ning Wang

摘要

The widespread use of AI-generated text introduces ethical concerns surrounding plagiarism, bias, and authenticity. Traditional detection systems, such as classifier-based approaches and linguistic feature extraction methods, have been developed to identify AI-generated content in domains like code, images, and text. However, these methods often suffer from limited generalizability across domains and decreasing accuracy as AI models become more fluent and human-like. Most current approaches evaluate AI output along a narrow set of dimensions, typically focusing on surface-level features such as fluency, syntax, or token frequency. This paper proposes a structured and multidimensional framework to evaluate AI-generated versus human-authored text across six key linguistic dimensions: grammatical consistency, vocabulary diversity, emotional expression, personalization, sensitivity to controversial topics, and response consistency. Using the HC3 dataset of 24,000 paired responses, we analyzed the differences between human and AI-generated text. Results show that human responses exhibit greater emotional diversity, vocabulary richness, and personalization, while AI outperforms grammatical accuracy and consistency. These findings contribute to a more holistic assessment of AI language models and highlight the need for evaluation methods that extend beyond traditional binary classification or syntax-based scoring.