Scholarlink

<p>This study aimed to evaluate the clinical safety, informational completeness, and patient-centered language of responses generated by four large language model (LLM)–based systems to standardized periodontal complaint-based queries. Seven standardized symptom-based periodontal queries were developed based on commonly reported patient complaints and submitted in Turkish to Copilot, Gemini, Claude, and ChatGPT. Responses were evaluated using a structured rule-based framework consisting of a Content Coverage Score (CCS), Risk of Harm Score (RHS), and Patient Language Score (PLS). Assessments were performed by two blinded human reviewers and one blinded AI-based evaluator. Human consensus, AI consensus, and combined evaluator scores were calculated. Agreement between evaluators was assessed using intraclass correlation coefficients (ICC), while human–AI differences and inter-model comparisons were analyzed using non-parametric statistical tests. Excellent agreement was observed between human reviewers, repeated AI evaluations, and human–AI consensus scores (ICC range: 0.911–0.925; <i>p</i> &lt; 0.001). No significant difference was found between overall human and AI consensus scores (<i>p</i> = 0.985). Across the four AI systems, no statistically significant differences were observed in CCS or PLS scores in the human, AI, or combined evaluator analyses (all <i>p</i> &gt; 0.05). PLS scores were generally high across models, indicating good linguistic accessibility for patients. No clearly harmful guidance was identified by the human reviewers. Overall, LLM-based systems generated clinically safe, reasonably comprehensive, and generally patient-accessible responses to common periodontal complaint-based queries. Although these systems may serve as supplementary sources of periodontal health information, they cannot replace individualized clinical evaluation and professional dental consultation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Clinical safety, content coverage, and patient-centered language of AI responses to periodontal complaint-based queries: a comparative study of four large language models

  • 期刊论文

  • 发表日期：

    2026年7月7日

查看全文
  • Çağrı Esen,
  • Mehmet Gül,
  • Fatih Karayürek

摘要

This study aimed to evaluate the clinical safety, informational completeness, and patient-centered language of responses generated by four large language model (LLM)–based systems to standardized periodontal complaint-based queries. Seven standardized symptom-based periodontal queries were developed based on commonly reported patient complaints and submitted in Turkish to Copilot, Gemini, Claude, and ChatGPT. Responses were evaluated using a structured rule-based framework consisting of a Content Coverage Score (CCS), Risk of Harm Score (RHS), and Patient Language Score (PLS). Assessments were performed by two blinded human reviewers and one blinded AI-based evaluator. Human consensus, AI consensus, and combined evaluator scores were calculated. Agreement between evaluators was assessed using intraclass correlation coefficients (ICC), while human–AI differences and inter-model comparisons were analyzed using non-parametric statistical tests. Excellent agreement was observed between human reviewers, repeated AI evaluations, and human–AI consensus scores (ICC range: 0.911–0.925; p < 0.001). No significant difference was found between overall human and AI consensus scores (p = 0.985). Across the four AI systems, no statistically significant differences were observed in CCS or PLS scores in the human, AI, or combined evaluator analyses (all p > 0.05). PLS scores were generally high across models, indicating good linguistic accessibility for patients. No clearly harmful guidance was identified by the human reviewers. Overall, LLM-based systems generated clinically safe, reasonably comprehensive, and generally patient-accessible responses to common periodontal complaint-based queries. Although these systems may serve as supplementary sources of periodontal health information, they cannot replace individualized clinical evaluation and professional dental consultation.