Scholarlink

Background <p>Large Language Models (LLMs) such as ChatGPT and Google Gemini are increasingly explored for their potential in patient education, particularly in the perioperative setting. As text-based tools trained on extensive datasets, they can generate detailed responses to common medical queries. However, their utility in delivering reliable, understandable, and emotionally appropriate preanesthetic information remains unclear.</p> Methods <p>We conducted a prospective observational study comparing ChatGPT and Google Gemini in generating educational content for patients undergoing laparoscopic cholecystectomy. From 68 patient questions submitted by anesthesiologists, 13 high-relevance items were selected. Responses from both models were independently rated by 20 anesthesiologists using a 5-point Likert scale across four domains: accuracy, comprehensiveness, clarity, and safety. Mixed-effects ordinal regression with random intercepts for rater and question estimated odds ratios (OR) and 95% confidence intervals (CI) for ChatGPT versus Gemini. Readability was assessed using standard linguistic indices, and sentiment analysis was performed. Inter-rater reliability was evaluated using Krippendorff’s α.</p> Results <p>ChatGPT had significantly higher odds of receiving better scores for accuracy (OR 2.32, 95% CI 1.62–3.32, <i>p</i> &lt; 0.001) and comprehensiveness (OR 2.38, 95% CI 1.67–3.37, <i>p</i> &lt; 0.001), with no differences for clarity (OR 1.05, 95% CI 0.75–1.47) or safety (OR 1.01, 95% CI 0.72–1.43). Gemini generated text with greater readability, demonstrated by a lower Flesch-Kincaid Grade level (<i>p</i> = 0.04) and higher Flesch Reading Ease score (<i>p</i> = 0.04). Sentiment analysis showed Gemini responses contained a wider emotional range, while ChatGPT responses were more neutral overall. Inter-rater reliability was low across domains (Krippendorff’s α 0.23–0.46).</p> Conclusion <p>ChatGPT produced more accurate and comprehensive perioperative anesthesia information, whereas Gemini offered greater readability and emotional expressiveness. Both models may serve as adjuncts in preanesthetic patient education but are not substitutes for clinician counselling. Larger, multi-center studies incorporating direct patient testing are warranted to validate these findings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Artificial intelligence in anesthesia: comparison of the utility of ChatGPT v/s google gemini large language models in pre-anesthetic education: content, readability and sentiment analysis

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月19日

查看全文
  • Prachi Sharma,
  • Bharti Sidhu,
  • Ashwini Reddy,
  • Puja Saxena,
  • Ritu Gupta,
  • Ravneet Kaur Gill,
  • Ananya Ray,
  • Rajeev Chauhan

摘要

Background

Large Language Models (LLMs) such as ChatGPT and Google Gemini are increasingly explored for their potential in patient education, particularly in the perioperative setting. As text-based tools trained on extensive datasets, they can generate detailed responses to common medical queries. However, their utility in delivering reliable, understandable, and emotionally appropriate preanesthetic information remains unclear.

Methods

We conducted a prospective observational study comparing ChatGPT and Google Gemini in generating educational content for patients undergoing laparoscopic cholecystectomy. From 68 patient questions submitted by anesthesiologists, 13 high-relevance items were selected. Responses from both models were independently rated by 20 anesthesiologists using a 5-point Likert scale across four domains: accuracy, comprehensiveness, clarity, and safety. Mixed-effects ordinal regression with random intercepts for rater and question estimated odds ratios (OR) and 95% confidence intervals (CI) for ChatGPT versus Gemini. Readability was assessed using standard linguistic indices, and sentiment analysis was performed. Inter-rater reliability was evaluated using Krippendorff’s α.

Results

ChatGPT had significantly higher odds of receiving better scores for accuracy (OR 2.32, 95% CI 1.62–3.32, p < 0.001) and comprehensiveness (OR 2.38, 95% CI 1.67–3.37, p < 0.001), with no differences for clarity (OR 1.05, 95% CI 0.75–1.47) or safety (OR 1.01, 95% CI 0.72–1.43). Gemini generated text with greater readability, demonstrated by a lower Flesch-Kincaid Grade level (p = 0.04) and higher Flesch Reading Ease score (p = 0.04). Sentiment analysis showed Gemini responses contained a wider emotional range, while ChatGPT responses were more neutral overall. Inter-rater reliability was low across domains (Krippendorff’s α 0.23–0.46).

Conclusion

ChatGPT produced more accurate and comprehensive perioperative anesthesia information, whereas Gemini offered greater readability and emotional expressiveness. Both models may serve as adjuncts in preanesthetic patient education but are not substitutes for clinician counselling. Larger, multi-center studies incorporating direct patient testing are warranted to validate these findings.