Scholarlink

Objective <p>To evaluate the accuracy, appropriateness, usefulness, and readability of artificial intelligence–generated postoperative counselling following functional endoscopic sinus surgery (FESS).</p> Methods <p>A descriptive cross-sectional study was conducted using a structured set of 15 postoperative counselling questions reflecting routine patient concerns after FESS. Questions were developed based on standard postoperative protocols and validated using the Content Validity Index. Each validated question was posed to ChatGPT (OpenAI; version accessed January 2026). Responses were independently assessed by four otorhinolaryngologists for appropriateness, accuracy, and usefulness using a 5-point Likert scale. Inter-rater reliability was analyzed using intraclass correlation coefficients (ICC). Readability was assessed using Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKGL) indices.</p> Results <p>The mean composite score across all domains was 3.91 ± 0.64. Appropriateness scored highest (4.12), followed by usefulness (4.02), while accuracy scored comparatively lower (3.59). Inter-rater reliability was excellent for appropriateness (ICC = 0.94), good for accuracy (ICC = 0.85), and moderate for usefulness (ICC = 0.69). Domain-wise analysis demonstrated higher scores for protocol-driven topics such as nasal hygiene and medication use, while lifestyle and recovery-related counselling showed greater variability. Readability analysis revealed a mean FRE score of 41.2 and an FKGL of 12.8, indicating content written at a senior-school to college reading level.</p> Conclusion <p>AI-generated postoperative counselling after FESS demonstrates acceptable clinical reliability for standardized guidance but shows limitations in contextual accuracy and linguistic accessibility. ChatGPT may serve as a supplementary educational tool; however, clinician-led counselling remains essential.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Reliability and clinical accuracy of ChatGPT in post-FESS counselling: a multi-reviewer evaluation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月14日

查看全文
  • Paramita Debnath,
  • Misbahul Haque,
  • Prakriti Samaddar,
  • Neha Yadav

摘要

Objective

To evaluate the accuracy, appropriateness, usefulness, and readability of artificial intelligence–generated postoperative counselling following functional endoscopic sinus surgery (FESS).

Methods

A descriptive cross-sectional study was conducted using a structured set of 15 postoperative counselling questions reflecting routine patient concerns after FESS. Questions were developed based on standard postoperative protocols and validated using the Content Validity Index. Each validated question was posed to ChatGPT (OpenAI; version accessed January 2026). Responses were independently assessed by four otorhinolaryngologists for appropriateness, accuracy, and usefulness using a 5-point Likert scale. Inter-rater reliability was analyzed using intraclass correlation coefficients (ICC). Readability was assessed using Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKGL) indices.

Results

The mean composite score across all domains was 3.91 ± 0.64. Appropriateness scored highest (4.12), followed by usefulness (4.02), while accuracy scored comparatively lower (3.59). Inter-rater reliability was excellent for appropriateness (ICC = 0.94), good for accuracy (ICC = 0.85), and moderate for usefulness (ICC = 0.69). Domain-wise analysis demonstrated higher scores for protocol-driven topics such as nasal hygiene and medication use, while lifestyle and recovery-related counselling showed greater variability. Readability analysis revealed a mean FRE score of 41.2 and an FKGL of 12.8, indicating content written at a senior-school to college reading level.

Conclusion

AI-generated postoperative counselling after FESS demonstrates acceptable clinical reliability for standardized guidance but shows limitations in contextual accuracy and linguistic accessibility. ChatGPT may serve as a supplementary educational tool; however, clinician-led counselling remains essential.