Scholarlink

Background <p>Molar incisor hypomineralization (MIH) is a clinically challenging developmental enamel defect that requires accurate diagnosis and nuanced management decisions. Although large language models (LLMs) are increasingly used as sources of clinical information in dentistry, the quality, consistency, and reference accuracy of their MIH-related explanations remain largely unexplored. To date, no study has systematically compared the clinical quality or citation accuracy of LLM-generated MIH information. This study comparatively evaluated how two widely used LLMs generate clinical information across repeated sessions.</p> Methods <p>Twenty open-ended MIH questions were developed in accordance with current clinical guidelines and organized into four categories: diagnosis, etiology, treatment, and differential diagnosis. ChatGPT-4o and DeepSeek R1 were each prompted with all questions during three independent sessions (morning, afternoon, evening) on the same day, generating a total of 120 responses. All responses were anonymized and evaluated by 24 calibrated pediatric dentists using the five-point Global Quality Scale (GQS). References provided by the models were independently verified by two reviewers and categorized as real or fabricated. Statistical analyses included Shapiro–Wilk tests, paired t-tests, Wilcoxon signed-rank tests, repeated-measures ANOVA, Friedman tests, Holm-corrected post-hoc comparisons, and ICC(2,1), with significance set at <i>p</i> &lt; 0.05.</p> Results <p>Across all time points and all four MIH-related categories, DeepSeek R1 consistently achieved significantly higher GQS scores than ChatGPT-4o (all adjusted <i>p</i> &lt; 0.005). Mean score differences ranged from + 0.36 to + 0.71, with the largest gap observed for etiology questions in the evening session. When overall scores were examined, DeepSeek R1 (4.44 ± 0.54) again outperformed ChatGPT-4o (3.99 ± 0.59) (t(23) = 11.83, <i>p</i> &lt; 0.0001). Both models showed statistically significant but clinically small session-related variations, with acceptable reliability indicated by ICC values (0.72 for ChatGPT-4o; 0.77 for DeepSeek R1). Reference verification revealed notable fabrication rates in both models: ChatGPT-4o provided 46.5% real and 53.5% fake references, while DeepSeek R1 provided 34.2% real and 65.8% fake references.</p> Conclusions <p>DeepSeek R1 and ChatGPT-4o each demonstrated distinct strengths in generating MIH-related clinical explanations, with DeepSeek providing more detailed and context-focused responses and ChatGPT-4o producing clearer, more structured overviews. Although the score differences were modest, they reflect meaningful variations when applied to a condition as diagnostically complex as MIH. Both models showed acceptable temporal stability; however, their substantial rates of fabricated references underscore the need for careful expert oversight. Overall, while LLMs may support early learning, patient communication, or preliminary clinical orientation, neither model currently meets the accuracy or citation standards required for autonomous clinical use in pediatric dentistry.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Comparative assessment of quality, consistency, and reference accuracy of MIH-related clinical information generated by ChatGPT-4o and DeepSeek R1

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月7日

查看全文
  • Zübeyde Uçar Gündoğar,
  • Derya Sarıoğlu

摘要

Background

Molar incisor hypomineralization (MIH) is a clinically challenging developmental enamel defect that requires accurate diagnosis and nuanced management decisions. Although large language models (LLMs) are increasingly used as sources of clinical information in dentistry, the quality, consistency, and reference accuracy of their MIH-related explanations remain largely unexplored. To date, no study has systematically compared the clinical quality or citation accuracy of LLM-generated MIH information. This study comparatively evaluated how two widely used LLMs generate clinical information across repeated sessions.

Methods

Twenty open-ended MIH questions were developed in accordance with current clinical guidelines and organized into four categories: diagnosis, etiology, treatment, and differential diagnosis. ChatGPT-4o and DeepSeek R1 were each prompted with all questions during three independent sessions (morning, afternoon, evening) on the same day, generating a total of 120 responses. All responses were anonymized and evaluated by 24 calibrated pediatric dentists using the five-point Global Quality Scale (GQS). References provided by the models were independently verified by two reviewers and categorized as real or fabricated. Statistical analyses included Shapiro–Wilk tests, paired t-tests, Wilcoxon signed-rank tests, repeated-measures ANOVA, Friedman tests, Holm-corrected post-hoc comparisons, and ICC(2,1), with significance set at p < 0.05.

Results

Across all time points and all four MIH-related categories, DeepSeek R1 consistently achieved significantly higher GQS scores than ChatGPT-4o (all adjusted p < 0.005). Mean score differences ranged from + 0.36 to + 0.71, with the largest gap observed for etiology questions in the evening session. When overall scores were examined, DeepSeek R1 (4.44 ± 0.54) again outperformed ChatGPT-4o (3.99 ± 0.59) (t(23) = 11.83, p < 0.0001). Both models showed statistically significant but clinically small session-related variations, with acceptable reliability indicated by ICC values (0.72 for ChatGPT-4o; 0.77 for DeepSeek R1). Reference verification revealed notable fabrication rates in both models: ChatGPT-4o provided 46.5% real and 53.5% fake references, while DeepSeek R1 provided 34.2% real and 65.8% fake references.

Conclusions

DeepSeek R1 and ChatGPT-4o each demonstrated distinct strengths in generating MIH-related clinical explanations, with DeepSeek providing more detailed and context-focused responses and ChatGPT-4o producing clearer, more structured overviews. Although the score differences were modest, they reflect meaningful variations when applied to a condition as diagnostically complex as MIH. Both models showed acceptable temporal stability; however, their substantial rates of fabricated references underscore the need for careful expert oversight. Overall, while LLMs may support early learning, patient communication, or preliminary clinical orientation, neither model currently meets the accuracy or citation standards required for autonomous clinical use in pediatric dentistry.