Artificial intelligence in emergency and trauma radiology: ASER AI/ML expert panel Delphi consensus statement on research guidelines, practices, and priorities

Background

Emergency/trauma radiology artificial intelligence (AI) is maturing along all stages of technology readiness, with research and development (R&D) ranging from data curation and algorithm development to post-market monitoring and retraining.

Purpose

To develop an expert consensus document on best research practices and methodological priorities for emergency/trauma radiology AI.

Methods

A Delphi consensus exercise was conducted by the ASER AI/ML expert panel between 2022-2024. In phase 1, a steering committee (7 panelists) established key themes- curation; validity; human factors; workflow; barriers; future avenues; and ethics- and generated an edited, collated long-list of statements. In phase 2, two Delphi rounds using anonymous RAND/UCLA Likert grading were conducted with web-based data capture (round 1) and a bespoke excel document with literature hyperlinks (round 2). Between rounds, editing and knowledge synthesis helped maximize consensus. Statements reaching ≥80% agreement were included in the final document.

Results

Delphi rounds 1 and 2 consisted of 81 and 78 items, respectively.18/21 expert panelists (86%) responded to round 1, and 15 to round 2 (17% drop-out). Consensus was reached for 65 statements. Observations were summarized and contextualized. Statements with unanimous consensus centered around transparent methodologic reporting; testing for generalizability and robustness with external data; and benchmarking performance with appropriate metrics and baselines. A manuscript draft was circulated to panelists for editing and final approval.

Conclusions

The document is meant as a framework to foster best-practices and further discussion among researchers working on various aspects of emergency and trauma radiology AI.

Graphical abstract