<p>ChatGPT has seen increased use by caregivers as a source of medical information. Pediatric neurology has added complexities because there are various disorders, changing diagnoses and potential consequences of misinformation. To evaluate the accuracy, comprehensiveness and reproducibility of ChatGPT responses to frequently asked pediatric neurology questions. In total, 105 unique questions across seven pediatric neurology subspecialties were included. Every question was input twice into ChatGPT (GPT-5) during separate sessions. Two certified pediatric neurologists reviewed the responses and rated them on a 4-point scale: 1-complete and accurate,2- correct but not complete,3- partially incorrect,4-not correct. Reproducibility was considered to be agreement between grading categories (grade 1–2 and grade 3–4) throughout repeated responses. 60.95%(64/105) of responses were rated as comprehensive, while 31.43%(33/105) were correct but incomplete; fully incorrect responses were uncommon 3.8%(4/105). Accuracy and reproducibility were highest for structured conditions such as cerebral palsy and movement disorders, and lower for more heterogeneous domains including neurodevelopmental delay and epilepsy. Overall reproducibility was high 95.2%(100/105). ChatGPT mostly delivers accurate information in pediatric neurology, with variable depth of response depending on the disorder. It is most appropriate for caregiver education with oversight by the clinician.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Assessing the accuracy of ChatGPT responses to caregiver questions in pediatric neurology: an observational study

  • Miral A. Almomani,
  • Basima A. Almomani,
  • Hashim Al-Qudah,
  • Rawan Abed Alrahman,
  • Seleina O. El-bawaneh,
  • Sondos Hawari,
  • Ahmad A. Alqudah

摘要

ChatGPT has seen increased use by caregivers as a source of medical information. Pediatric neurology has added complexities because there are various disorders, changing diagnoses and potential consequences of misinformation. To evaluate the accuracy, comprehensiveness and reproducibility of ChatGPT responses to frequently asked pediatric neurology questions. In total, 105 unique questions across seven pediatric neurology subspecialties were included. Every question was input twice into ChatGPT (GPT-5) during separate sessions. Two certified pediatric neurologists reviewed the responses and rated them on a 4-point scale: 1-complete and accurate,2- correct but not complete,3- partially incorrect,4-not correct. Reproducibility was considered to be agreement between grading categories (grade 1–2 and grade 3–4) throughout repeated responses. 60.95%(64/105) of responses were rated as comprehensive, while 31.43%(33/105) were correct but incomplete; fully incorrect responses were uncommon 3.8%(4/105). Accuracy and reproducibility were highest for structured conditions such as cerebral palsy and movement disorders, and lower for more heterogeneous domains including neurodevelopmental delay and epilepsy. Overall reproducibility was high 95.2%(100/105). ChatGPT mostly delivers accurate information in pediatric neurology, with variable depth of response depending on the disorder. It is most appropriate for caregiver education with oversight by the clinician.