Evaluating Perceived Workload, Usability and Usefulness of Artificial Intelligence Systems in Low-Resource Settings: Semi-automated Classification and Detection of Community Acquired Pneumonia
摘要
The use of Artificial Intelligence (AI) techniques in radiological workflows is increasingly becoming mainstream. However, the uptake of AI techniques is still low in low-resource settings such as the Global South. This paper presents a study conducted in a setting with low AI uptake, to determine the impact of AI on Radiologists’ workload when interpreting medical images. Two (2) AI models-a classification model and detection model indicating potential areas of interest-were implemented to facilitate the semi-automated interpretation of medical images for Pneumonia. In addition, a Web-based DICOM Viewer was implemented to interface the AI models. To determine the appropriate model configuration, two (2) experts-a Radiologist and Radiology Resident-participated in a focus group discussion aimed at determining how the AI models could facilitate interpretation processes. A comparative controlled experiment was subsequently conducted with 12 Radiology Residents at a large University Teaching Hospital, to assess the impact of AI on the workload and its perceived usefulness. NASA Task Load Index (TLX) and Technology Acceptance Model (TAM) 2 questionnaires were employed to measure the workload and usefulness. The results indicate that the perceived workload is significantly less when using the AI solution, with an overall NASA-TLX score of 1.86. Furthermore, the perceived usefulness of the AI solution is demonstrated through the positive responses for all the eight TAM 2 constructs. This study experimentally demonstrates the potential of utilising AI for the semi-automated interpretation of medical images in low-resource settings.