Scholarlink

Background <p>Amidst the current enthusiasm concerning artificial intelligence and its possible application in the composition of different kinds of scientific and non-scientific written documents, we evaluated the usage of artificial intelligence for writing surgical short reviews.</p> Methods <p>In order to assess the formal and content quality of AI-generated texts compared to human written texts, ten AI-based text generators (five chatbots and five content creators) and four surgeons in training received the same prompt for a short scientific article on a liver surgery theme. All texts were anonymized and subsequently evaluated by three experienced liver surgeons based on a pre-defined scoring scheme, as well as for quality of references and readability according to readability indices. Furthermore, all texts were tested for plagiarism using PlagScan.</p> Results <p>Overall percentage of correct assessment for AI/non-AI generation by experienced surgeons lay at 78.57%. Human written text had a mean word count of 1054 versus 874 in AI-generated text, with a higher mean Flesh Reading Ease Score (FRE, 26.2 ± 5.1 versus 17.7 ± 6.1). References were PubMed-listed in 100% for human versus 46% for AI-generated text, with only one non-human text reaching 100% formally correct citation of references. PlagScan found 6.4%±1.3 mean resemblance to existing texts for human versus 7.6%±4.5 for AI-generated text.</p> Discussion <p>Overall, AI could already mislead experienced scientific surgeons in 26.7% of cases into believing it to be human. However, formal requirements, especially considering referencing, are still in great need of improvement with only one of AI-generated articles fulfilling our quality requirements.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

The role of large language models in the writing of surgical reviews: fact or fantasy?

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月3日

查看全文
  • Daphne E. DeTemple,
  • Simon Störzer,
  • Sahar Arbabzadah,
  • Anna Riddermann,
  • Felix Gronau,
  • Kai Timrott,
  • Hüseyin Bektas,
  • Moritz Kleine

摘要

Background

Amidst the current enthusiasm concerning artificial intelligence and its possible application in the composition of different kinds of scientific and non-scientific written documents, we evaluated the usage of artificial intelligence for writing surgical short reviews.

Methods

In order to assess the formal and content quality of AI-generated texts compared to human written texts, ten AI-based text generators (five chatbots and five content creators) and four surgeons in training received the same prompt for a short scientific article on a liver surgery theme. All texts were anonymized and subsequently evaluated by three experienced liver surgeons based on a pre-defined scoring scheme, as well as for quality of references and readability according to readability indices. Furthermore, all texts were tested for plagiarism using PlagScan.

Results

Overall percentage of correct assessment for AI/non-AI generation by experienced surgeons lay at 78.57%. Human written text had a mean word count of 1054 versus 874 in AI-generated text, with a higher mean Flesh Reading Ease Score (FRE, 26.2 ± 5.1 versus 17.7 ± 6.1). References were PubMed-listed in 100% for human versus 46% for AI-generated text, with only one non-human text reaching 100% formally correct citation of references. PlagScan found 6.4%±1.3 mean resemblance to existing texts for human versus 7.6%±4.5 for AI-generated text.

Discussion

Overall, AI could already mislead experienced scientific surgeons in 26.7% of cases into believing it to be human. However, formal requirements, especially considering referencing, are still in great need of improvement with only one of AI-generated articles fulfilling our quality requirements.