Scholarlink

Background <p>Artificial intelligence (AI) chatbots are increasingly used in healthcare to provide health-related information and answer patient questions. However, their reliability in specialized dental fields such as restorative dentistry remains insufficiently evaluated. This study aimed to evaluate and compare the accuracy and consistency of responses generated by five artificial intelligence chatbot systems—ChatGPT-3.5 and ChatGPT-4 (OpenAI), Bing Chat (Microsoft), Gemini (Google), and Claude-Instant (Anthropic)—regarding dental bleaching.</p> Methods <p>Fifteen frequently asked questions about dental bleaching, identified by restorative dentistry specialists, were categorized as undergraduate-or specialist-level questions. All questions were submitted to five artificial intelligence chatbots in both Turkish and English. Each question was asked three times per day over three consecutive days using standardized prompts. Responses were independently evaluated by two experts using a five-point Likert scale, and mean scores were calculated. A three-way ANOVA was conducted to assess the effects of chatbot type, knowledge level, and question language on response accuracy. Inter-rater agreement between evaluators was assessed using Cohen’s kappa coefficient. Statistical significance was set at <i>p</i> &lt; 0.05.</p> Results <p>Chatbot type had a significant effect on response accuracy (<i>p</i> &lt; 0.001, η² = 0.405). ChatGPT-4 showed the highest accuracy, followed by Claude-Instant and GPT-3.5, whereas Gemini and especially Bing Chat demonstrated significantly lower performance (<i>p</i> &lt; 0.001). Question language and knowledge level showed no significant main effects (<i>p</i> &gt; 0.05). Significant interactions were observed between chatbot type and knowledge level and between chatbot type and language (<i>p</i> &lt; 0.001). No significant differences were observed across days or time periods (<i>p</i> &gt; 0.05).</p> Conclusions <p>The accuracy of chatbot-generated information regarding dental bleaching depends strongly on the specific AI model used. Advanced large language models, particularly ChatGPT-4, generate more accurate and consistent responses than other evaluated systems. AI chatbots should therefore not be considered interchangeable sources of clinical information, and their outputs should be interpreted cautiously and verified with professional guidance.</p> Clinical relevance <p>These findings highlight the importance of critically evaluating AI-generated health information and emphasize that chatbot responses should not replace professional clinical consultation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Performance of AI chatbots in answering questions on tooth bleaching: a multilingual comparative study

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月16日

查看全文
  • Fehime Alkan Aygör,
  • Hasibe Sevilay Bahadır,
  • Ali Can Bulut,
  • Caner Öztürk

摘要

Background

Artificial intelligence (AI) chatbots are increasingly used in healthcare to provide health-related information and answer patient questions. However, their reliability in specialized dental fields such as restorative dentistry remains insufficiently evaluated. This study aimed to evaluate and compare the accuracy and consistency of responses generated by five artificial intelligence chatbot systems—ChatGPT-3.5 and ChatGPT-4 (OpenAI), Bing Chat (Microsoft), Gemini (Google), and Claude-Instant (Anthropic)—regarding dental bleaching.

Methods

Fifteen frequently asked questions about dental bleaching, identified by restorative dentistry specialists, were categorized as undergraduate-or specialist-level questions. All questions were submitted to five artificial intelligence chatbots in both Turkish and English. Each question was asked three times per day over three consecutive days using standardized prompts. Responses were independently evaluated by two experts using a five-point Likert scale, and mean scores were calculated. A three-way ANOVA was conducted to assess the effects of chatbot type, knowledge level, and question language on response accuracy. Inter-rater agreement between evaluators was assessed using Cohen’s kappa coefficient. Statistical significance was set at p < 0.05.

Results

Chatbot type had a significant effect on response accuracy (p < 0.001, η² = 0.405). ChatGPT-4 showed the highest accuracy, followed by Claude-Instant and GPT-3.5, whereas Gemini and especially Bing Chat demonstrated significantly lower performance (p < 0.001). Question language and knowledge level showed no significant main effects (p > 0.05). Significant interactions were observed between chatbot type and knowledge level and between chatbot type and language (p < 0.001). No significant differences were observed across days or time periods (p > 0.05).

Conclusions

The accuracy of chatbot-generated information regarding dental bleaching depends strongly on the specific AI model used. Advanced large language models, particularly ChatGPT-4, generate more accurate and consistent responses than other evaluated systems. AI chatbots should therefore not be considered interchangeable sources of clinical information, and their outputs should be interpreted cautiously and verified with professional guidance.

Clinical relevance

These findings highlight the importance of critically evaluating AI-generated health information and emphasize that chatbot responses should not replace professional clinical consultation.