Scholarlink

Background: <p>Simulated patient systems are vital in medical education and research, providing safe, integrative training environments and supporting clinical decision-making. Progressive Artificial Intelligence (AI) technologies, such as Large Language Models (LLM), could advance simulated patient systems by replicating medical conditions and patient-doctor interactions with high fidelity and low cost. However, effectiveness and trustworthiness remain challenging.</p> Methods: <p>We developed AIPatient, a simulated patient system powered by LLM-based AI agents. The system incorporates the Retrieval Augmented Generation (RAG) framework, powered by six task-specific LLM-based AI agents for complex reasoning. For simulation reality, the system is also powered by the AIPatient KG (Knowledge Graph), built with de-identified real patient data from the Medical Information Mart for Intensive Care (MIMIC)-III database.</p> Results: <p>Here we show that the system’s accuracy in Electronic Health Record (EHR)-based medical Question Answering (QA), readability, robustness, and stability. Specifically, the system achieves a QA accuracy of 94.15% when all six agents, surpassing benchmarks with partial or no agent integration. Its knowledgebase demonstrates high validity (F1 score=0.89). Readability scores show median Flesch Reading Ease at 68.77 and median Flesch Kincaid Grade at 6.4, indicating accessibility to all medical professionals. Robustness and stability are confirmed with non-significant variance (ANOVA F-value = 0.6126, p &gt; 0.1; F-value = 0.782, p &gt; 0.1). A user study with medical students shows that AIPatient delivers high fidelity, usability, and educational value, matching or exceeding human-simulated patients in history-taking.</p> Conclusions: <p>Large language model–based simulated patient systems provide accurate, readable, and reliable medical encounters and demonstrates potential to transform medical education.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月19日

查看全文
  • Huizi Yu,
  • Jiayan Zhou,
  • Lingyao Li,
  • Shan Chen,
  • Jack Gallifant,
  • Anye Shi,
  • Jie Sun,
  • Xiang Li,
  • Jingxian He,
  • Wenyue Hua,
  • Mingyu Jin,
  • Guang Chen,
  • Yang Zhou,
  • Zhao Li,
  • Trisha Gupte,
  • Ming-Li Chen,
  • Zahra Azizi,
  • Qi Dou,
  • Bryan P. Yan,
  • Yanqiu Xing,
  • Yongfeng Zhang,
  • Themistocles L. Assimes,
  • Danielle S. Bitterman,
  • Xin Ma,
  • Lin Lu,
  • Lizhou Fan

摘要

Background:

Simulated patient systems are vital in medical education and research, providing safe, integrative training environments and supporting clinical decision-making. Progressive Artificial Intelligence (AI) technologies, such as Large Language Models (LLM), could advance simulated patient systems by replicating medical conditions and patient-doctor interactions with high fidelity and low cost. However, effectiveness and trustworthiness remain challenging.

Methods:

We developed AIPatient, a simulated patient system powered by LLM-based AI agents. The system incorporates the Retrieval Augmented Generation (RAG) framework, powered by six task-specific LLM-based AI agents for complex reasoning. For simulation reality, the system is also powered by the AIPatient KG (Knowledge Graph), built with de-identified real patient data from the Medical Information Mart for Intensive Care (MIMIC)-III database.

Results:

Here we show that the system’s accuracy in Electronic Health Record (EHR)-based medical Question Answering (QA), readability, robustness, and stability. Specifically, the system achieves a QA accuracy of 94.15% when all six agents, surpassing benchmarks with partial or no agent integration. Its knowledgebase demonstrates high validity (F1 score=0.89). Readability scores show median Flesch Reading Ease at 68.77 and median Flesch Kincaid Grade at 6.4, indicating accessibility to all medical professionals. Robustness and stability are confirmed with non-significant variance (ANOVA F-value = 0.6126, p > 0.1; F-value = 0.782, p > 0.1). A user study with medical students shows that AIPatient delivers high fidelity, usability, and educational value, matching or exceeding human-simulated patients in history-taking.

Conclusions:

Large language model–based simulated patient systems provide accurate, readable, and reliable medical encounters and demonstrates potential to transform medical education.