Generative artificial intelligence, such as large language models (LLMs), has demonstrated remarkable capabilities in text generation and holds potential for significant productivity enhancement when integrated with tools. However, LLMs come with inherent architectural limitations, leading to the phenomenon of “hallucinations” during text generation. This issue presents several challenges, among which the problem of authorship attribution for generated texts has become a critical topic, attracting widespread attention. In this research, we conducted a comprehensive evaluation comparing the performance of both closed-source and open-source LLMs with human experts in authorship attribution tasks. Through multiple rewriting experiments and cross-detection trials, our findings suggest that while current LLMs cannot fully replace human expertise in authorship attribution, they are effective and feasible as assistive tools in this domain.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating Human-Large Language Model Alignment in Authorship Attribution

  • Yidong He,
  • Hongzhen Du

摘要

Generative artificial intelligence, such as large language models (LLMs), has demonstrated remarkable capabilities in text generation and holds potential for significant productivity enhancement when integrated with tools. However, LLMs come with inherent architectural limitations, leading to the phenomenon of “hallucinations” during text generation. This issue presents several challenges, among which the problem of authorship attribution for generated texts has become a critical topic, attracting widespread attention. In this research, we conducted a comprehensive evaluation comparing the performance of both closed-source and open-source LLMs with human experts in authorship attribution tasks. Through multiple rewriting experiments and cross-detection trials, our findings suggest that while current LLMs cannot fully replace human expertise in authorship attribution, they are effective and feasible as assistive tools in this domain.