How AI should be used in radiology: assessing ambiguity and completeness of intended use statements of commercial AI products

Background

Intended use statements (IUSs) are mandatory to obtain regulatory clearance for artificial intelligence (AI)-based medical devices in the European Union. In order to guide the safe use of AI-based medical devices, IUSs need to contain comprehensive and understandable information. This study analyzes the IUSs of CE-marked AI products listed on AIforRadiology.com for ambiguity and completeness.

Methods

We retrieved 157 IUSs of CE-marked AI products listed on AIforRadiology.com in September 2022. Duplicate products (n = 1), discontinued products (n = 3), and duplicate statements (n = 14) were excluded. The resulting IUSs were assessed for the presence of 6 items: medical indication, part of the body, patient population, user profile, use environment, and operating principle. Disclaimers, defined as contra-indications or warnings in the IUS, were identified and compared with claims.

Results

Of 139 AI products, the majority (n = 78) of IUSs mentioned 3 or less items. IUSs of only 7 products mentioned all 6 items. The intended body part (n = 115) and the operating principle (n = 116) were the most frequently mentioned components, while the intended use environment (n = 24) and intended patient population (n = 29) were mentioned less frequently. Fifty-six statements contained disclaimers that conflicted with the claims in 13 cases.

Conclusion

The majority of IUSs of CE-marked AI-based medical devices lack substantial information and, in few cases, contradict the claims of the product.

Critical relevance statement

To ensure correct usage and to avoid off-label use or foreseeable misuse of AI-based medical devices in radiology, manufacturers are encouraged to provide more comprehensive and less ambiguous intended use statements.

Key points

• Radiologists must know AI products’ intended use to avoid off-label use or misuse.

• Ninety-five percent (n = 132/139) of the intended use statements analyzed were incomplete.

• Nine percent (n = 13) of the intended use statements held disclaimers contradicting the claim of the AI product.

• Manufacturers and regulatory bodies must ensure that intended use statements are comprehensive.

Graphical Abstract