Objectives The efficacy of an MRI-based deep learning algorithm (DLA) for detecting acute ischemic stroke (AIS) was evaluated across readers with diverse medical backgrounds, because DLA performance may be user-dependent. Materials and methods This retrospective, multi-reader, multi-case crossover study included 407 MRI scans obtained from a single institution between April and June 2021. Nine readers with different backgrounds— radiology residents (1–2 years of radiology training), clinicians (no radiology training), and board-certified non-neuroradiologists (completed residency training)—independently read MRI scans, both with and without DLA detection probability. The ground truth was established by consensus among three neuroradiologists. The area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, diagnostic confidence (0–4), and inter-reader agreement were compared between the groups with and without DLA. Results In total, 407 patients (mean age, 66 years ± 16; 200 men) with 95 AIS (23%) were evaluated. Clinicians had the lowest baseline performance scores. The DLA significantly improved clinicians’ AUC (from 0.90 [95% CI: 0.82–0.99]; to 0.93 [0.87–0.99]; p < 0.01), sensitivity (from 0.77 [0.65–0.88]; to 0.88 [0.75–0.99]; p < 0.01), and diagnostic confidence (from 0.71 ± 1.42; to 0.83 ± 1.53; p < 0.01), and all readers’ inter-reader agreement (p < 0.01). Specificity for clinicians (from 0.95 [0.86–0.99] to 0.93 [0.80–0.99]; p = 0.55) and the performance of residents and non-neuroradiologists were not significantly affected by DLA assistance. Conclusion The DLA significantly improved the performance and diagnostic confidence of clinicians, the lowest-performing readers, and the inter-reader agreement of all readers in diagnosing AIS. Key Points <Emphasis Type="BoldItalic">Question</Emphasis> What is the efficacy of an MRI-based deep learning algorithm in assisting various medical professionals in identifying acute ischemic stroke? <Emphasis Type="BoldItalic">Findings</Emphasis> Among radiology residents, clinicians, and board-certified non-neuroradiologists, the algorithm significantly improved clinicians’ performance and diagnostic confidence while also enhancing inter-reader agreement for all readers. <Emphasis Type="BoldItalic">Clinical relevance</Emphasis> The deep learning algorithm significantly improves the detection performance and diagnostic confidence of clinicians, the lowest-performing readers, for acute ischemic stroke. Furthermore, the inter-reader agreement among various medical professionals has improved significantly. Graphical Abstract

Efficacy of MRI-based deep learning algorithm for detecting acute ischemic stroke: evaluation among diverse readers

期刊论文
发表日期：

2025年11月17日

查看全文

Jimin Kim,
Se Won Oh,
Ha Young Lee,
Sheen-Woo Lee,
Sungjun Hwang,
Heiko Meyer,
Stefan Huwer,
Gengyan Zhao,
Eli Gibson,
Dongyeob Han

摘要

Objectives

The efficacy of an MRI-based deep learning algorithm (DLA) for detecting acute ischemic stroke (AIS) was evaluated across readers with diverse medical backgrounds, because DLA performance may be user-dependent.

Materials and methods

This retrospective, multi-reader, multi-case crossover study included 407 MRI scans obtained from a single institution between April and June 2021. Nine readers with different backgrounds— radiology residents (1–2 years of radiology training), clinicians (no radiology training), and board-certified non-neuroradiologists (completed residency training)—independently read MRI scans, both with and without DLA detection probability. The ground truth was established by consensus among three neuroradiologists. The area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, diagnostic confidence (0–4), and inter-reader agreement were compared between the groups with and without DLA.

Results

In total, 407 patients (mean age, 66 years ± 16; 200 men) with 95 AIS (23%) were evaluated. Clinicians had the lowest baseline performance scores. The DLA significantly improved clinicians’ AUC (from 0.90 [95% CI: 0.82–0.99]; to 0.93 [0.87–0.99]; p < 0.01), sensitivity (from 0.77 [0.65–0.88]; to 0.88 [0.75–0.99]; p < 0.01), and diagnostic confidence (from 0.71 ± 1.42; to 0.83 ± 1.53; p < 0.01), and all readers’ inter-reader agreement (p < 0.01). Specificity for clinicians (from 0.95 [0.86–0.99] to 0.93 [0.80–0.99]; p = 0.55) and the performance of residents and non-neuroradiologists were not significantly affected by DLA assistance.

Conclusion

The DLA significantly improved the performance and diagnostic confidence of clinicians, the lowest-performing readers, and the inter-reader agreement of all readers in diagnosing AIS.

Key Points

Question What is the efficacy of an MRI-based deep learning algorithm in assisting various medical professionals in identifying acute ischemic stroke?

Findings Among radiology residents, clinicians, and board-certified non-neuroradiologists, the algorithm significantly improved clinicians’ performance and diagnostic confidence while also enhancing inter-reader agreement for all readers.

Clinical relevance The deep learning algorithm significantly improves the detection performance and diagnostic confidence of clinicians, the lowest-performing readers, for acute ischemic stroke. Furthermore, the inter-reader agreement among various medical professionals has improved significantly.

Graphical Abstract