Scholarlink

Background <p>Artificial intelligence (AI)-based chatbots are increasingly used by parents as convenient and fast-access sources of information on health-related topics. This study aimed to assess the readability, accuracy and overall quality of responses provided by ChatGPT-4o, Google Gemini and Microsoft Copilot to questions concerning deleterious oral habits in children.</p> Methods <p>A total of 43 questions, derived from real-life discussions on the Reddit platform, were revised for clarity and demographic diversity. These were classified into seven categories based on specific types of deleterious oral habits, including thumb sucking, bruxism, pacifier use, bruxism, tongue thrusting, lip sucking, nail biting, and mouth breathing. Responses from each AI chatbot were evaluated using multiple evaluation tools including Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), the modified DISCERN tool (mDISCERN), Global Quality Score (GQS), and misinformation scoring system. Statistical analyses were performed using the Kruskal–Wallis test followed by Dunn’s post hoc test for non-normally distributed variables, and one-way ANOVA with Tukey’s post hoc test for normally distributed variables (<i>p</i> &lt; .05).</p> Results <p>ChatGPT-4o generated responses with significantly lower readability and higher textual complexity compared to Gemini and Copilot, as reflected by its lower FRE (<i>p</i> = .0022) and higher FKGL (<i>p</i> = .0062) scores. ChatGPT-4o had 76.74% of its responses rated as excellent quality (GQS score of 5), compared to 44.19% for Gemini and 30.23% for Copilot. In terms of accuracy, ChatGPT-4o provided correct information for 93% of the questions (misinformation scores of 4 or 5), while Gemini and Copilot achieved 88.34% and 81.4%, respectively. Google Gemini achieved the highest mDISCERN score (34.1) due to better source referencing.</p> Conclusion <p>AI chatbots may serve as supplementary tools for parental education on oral health, yet their performance varies by platform. ChatGPT-4o excelled in accuracy and structure, Gemini in transparency, and Copilot in simplicity. However, these tools should not substitute for professional dental guidance. Enhancing readability and source referencing remains essential for improving the reliability of AI-generated health information.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluation of the readability, quality, and accuracy of AI chatbot responses to questions about deleterious oral habits

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月21日

查看全文
  • Ömer Tarık Özdemir,
  • Melis Yazır Kavan,
  • Yeliz Güven

摘要

Background

Artificial intelligence (AI)-based chatbots are increasingly used by parents as convenient and fast-access sources of information on health-related topics. This study aimed to assess the readability, accuracy and overall quality of responses provided by ChatGPT-4o, Google Gemini and Microsoft Copilot to questions concerning deleterious oral habits in children.

Methods

A total of 43 questions, derived from real-life discussions on the Reddit platform, were revised for clarity and demographic diversity. These were classified into seven categories based on specific types of deleterious oral habits, including thumb sucking, bruxism, pacifier use, bruxism, tongue thrusting, lip sucking, nail biting, and mouth breathing. Responses from each AI chatbot were evaluated using multiple evaluation tools including Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), the modified DISCERN tool (mDISCERN), Global Quality Score (GQS), and misinformation scoring system. Statistical analyses were performed using the Kruskal–Wallis test followed by Dunn’s post hoc test for non-normally distributed variables, and one-way ANOVA with Tukey’s post hoc test for normally distributed variables (p < .05).

Results

ChatGPT-4o generated responses with significantly lower readability and higher textual complexity compared to Gemini and Copilot, as reflected by its lower FRE (p = .0022) and higher FKGL (p = .0062) scores. ChatGPT-4o had 76.74% of its responses rated as excellent quality (GQS score of 5), compared to 44.19% for Gemini and 30.23% for Copilot. In terms of accuracy, ChatGPT-4o provided correct information for 93% of the questions (misinformation scores of 4 or 5), while Gemini and Copilot achieved 88.34% and 81.4%, respectively. Google Gemini achieved the highest mDISCERN score (34.1) due to better source referencing.

Conclusion

AI chatbots may serve as supplementary tools for parental education on oral health, yet their performance varies by platform. ChatGPT-4o excelled in accuracy and structure, Gemini in transparency, and Copilot in simplicity. However, these tools should not substitute for professional dental guidance. Enhancing readability and source referencing remains essential for improving the reliability of AI-generated health information.