Scholarlink

Objective <p>Documentation of myocardial ischemia prior to invasive coronary angiography is recommended to minimize patient risk. However, obtaining this information for quality-of-care assessment often requires extracting clinical information from unstructured electronic medical records text. To this end, we sought to evaluate multiple natural language processing (NLP) systems in their ability to classify cardiac stress test reports as documenting ischemia or no ischemia, implementing the one with the best combination of accuracy and feasibility.</p> Results <p>Four BERT large language models (LLMs) were fine-tuned, and a rules-based system was designed by training, validating, and testing on an annotated sample of 654 stress test reports from a multisite and multiyear dataset from the Veterans Health Administration (VHA). The LLM with the highest performance was a ClinicalBERT with precision, recall, and F1 of 86.4%, 100%, and 92.7%, respectively. The rules-based NLP system achieved similar results of 88.1%, 97.4%, and 92.5%, respectively. Stress test reports totaling 1,692,171 and representing 1,096,341 unique patients were classified using the rules-based system after ascertaining current technological limitations, and the system is presently operational for care quality evaluations. Utilizing NLP allows for accurate, high-throughput analysis of cardiac stress test text reports.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Utilizing large language models and natural language processing to classify ischemia status from cardiac stress tests in a large multicenter healthcare system

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月3日

查看全文
  • Shayna Cave,
  • Kelly S. Peterson,
  • Mary E. Plomondon,
  • Stephen W. Waldo

摘要

Objective

Documentation of myocardial ischemia prior to invasive coronary angiography is recommended to minimize patient risk. However, obtaining this information for quality-of-care assessment often requires extracting clinical information from unstructured electronic medical records text. To this end, we sought to evaluate multiple natural language processing (NLP) systems in their ability to classify cardiac stress test reports as documenting ischemia or no ischemia, implementing the one with the best combination of accuracy and feasibility.

Results

Four BERT large language models (LLMs) were fine-tuned, and a rules-based system was designed by training, validating, and testing on an annotated sample of 654 stress test reports from a multisite and multiyear dataset from the Veterans Health Administration (VHA). The LLM with the highest performance was a ClinicalBERT with precision, recall, and F1 of 86.4%, 100%, and 92.7%, respectively. The rules-based NLP system achieved similar results of 88.1%, 97.4%, and 92.5%, respectively. Stress test reports totaling 1,692,171 and representing 1,096,341 unique patients were classified using the rules-based system after ascertaining current technological limitations, and the system is presently operational for care quality evaluations. Utilizing NLP allows for accurate, high-throughput analysis of cardiac stress test text reports.