Scholarlink

Purpose <p>Artificial intelligence (AI) chatbots, such as ChatGPT-4, allow a user to ask questions on an interactive level. This study evaluated the correctness and completeness of responses to questions about ovarian cancer from a GPT-4 chatbot, LilyBot, compared with responses from healthcare professionals in gynecologic cancer care.</p> Methods <p>Fifteen categories of questions about ovarian cancer were collected from an online patient Chatgroup forum. Ten healthcare professionals in gynecologic oncology generated 150 questions and responses relative to these topics. Responses from LilyBot and the healthcare professionals were scored for correctness and completeness by eight independent healthcare professionals with similar backgrounds blinded to the identity of the responders. Differences between groups were analyzed with Mann–Whitney <i>U</i> and Kruskal–Wallis tests, followed by Tukey’s post hoc comparisons.</p> Results <p>Mean scores for overall performance for all 150 questions were significantly higher for LilyBot compared with the healthcare professionals for correctness (5.31 ± 0.98 vs. 5.07 ± 1.00, <i>p</i> = 0.017; range = 1–6) and completeness (2.66 ± 0.55 vs. 2.36 ± 0.55, <i>p</i> &lt; 0.001; range = 1–3). LilyBot had significantly higher scores for immunotherapy compared with the healthcare professionals for correctness (6.00 ± 0.00 vs. 4.70 ± 0.48, <i>p</i> = 0.020) and completeness (3.00 ± 0.00 vs. 2.00 ± 0.00,<i> p</i> &lt; 0.010); and gene therapy for completeness (3.00 ± 0.00 vs. 2.20 ± 0.42, <i>p</i> = 0.023).</p> Conclusions <p>The significantly better performance by LilyBot compared with healthcare professionals highlights the potential of ChatGPT-4-based dialogue systems to provide patients with clinical information about ovarian cancer.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AI-driven patient support: Evaluating the effectiveness of ChatGPT-4 in addressing queries about ovarian cancer compared with healthcare professionals in gynecologic oncology

  • 期刊论文

  • 发表日期：

    2025年4月1日

查看全文
  • Hung-Hsueh Chou,
  • Yi Hua Chen,
  • Chiu-Tzu Lin,
  • Hsien-Tsung Chang,
  • An-Chieh Wu,
  • Jia-Ling Tsai,
  • Hsiao-Wei Chen,
  • Ching-Chun Hsu,
  • Shu-Ya Liu,
  • Jian Tao Lee

摘要

Purpose

Artificial intelligence (AI) chatbots, such as ChatGPT-4, allow a user to ask questions on an interactive level. This study evaluated the correctness and completeness of responses to questions about ovarian cancer from a GPT-4 chatbot, LilyBot, compared with responses from healthcare professionals in gynecologic cancer care.

Methods

Fifteen categories of questions about ovarian cancer were collected from an online patient Chatgroup forum. Ten healthcare professionals in gynecologic oncology generated 150 questions and responses relative to these topics. Responses from LilyBot and the healthcare professionals were scored for correctness and completeness by eight independent healthcare professionals with similar backgrounds blinded to the identity of the responders. Differences between groups were analyzed with Mann–Whitney U and Kruskal–Wallis tests, followed by Tukey’s post hoc comparisons.

Results

Mean scores for overall performance for all 150 questions were significantly higher for LilyBot compared with the healthcare professionals for correctness (5.31 ± 0.98 vs. 5.07 ± 1.00, p = 0.017; range = 1–6) and completeness (2.66 ± 0.55 vs. 2.36 ± 0.55, p < 0.001; range = 1–3). LilyBot had significantly higher scores for immunotherapy compared with the healthcare professionals for correctness (6.00 ± 0.00 vs. 4.70 ± 0.48, p = 0.020) and completeness (3.00 ± 0.00 vs. 2.00 ± 0.00, p < 0.010); and gene therapy for completeness (3.00 ± 0.00 vs. 2.20 ± 0.42, p = 0.023).

Conclusions

The significantly better performance by LilyBot compared with healthcare professionals highlights the potential of ChatGPT-4-based dialogue systems to provide patients with clinical information about ovarian cancer.