Continuous Auditing Based Conformity Assessment (CABCA) is a new audit methodology developed to maintain the ongoing compliance of AI systems with various standards and industry-specific regulations. CABCA enables the operationalisation of those high-level requirements into measurable attributes to increase the level of automation. This paper presents CABCA and assesses its effectiveness through a proof-of-concept (PoC) implementation on a Medical Visual Question Answering (MedVQA) system, trained using the Radiology Objects in COntext (ROCO) dataset. In this evaluation, we partially apply guidelines from the “Artificial Intelligence (AI) in medical devices” questionnaire by the IG-NB, illustrating how to evaluate the conformity of an AI system to industry guidelines and standards.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Continuous Auditing Based Conformity Assessment for AI Systems: A Proof-of-Concept Evaluation

  • Dorian Knoblauch,
  • Abhishek Shrestha

摘要

Continuous Auditing Based Conformity Assessment (CABCA) is a new audit methodology developed to maintain the ongoing compliance of AI systems with various standards and industry-specific regulations. CABCA enables the operationalisation of those high-level requirements into measurable attributes to increase the level of automation. This paper presents CABCA and assesses its effectiveness through a proof-of-concept (PoC) implementation on a Medical Visual Question Answering (MedVQA) system, trained using the Radiology Objects in COntext (ROCO) dataset. In this evaluation, we partially apply guidelines from the “Artificial Intelligence (AI) in medical devices” questionnaire by the IG-NB, illustrating how to evaluate the conformity of an AI system to industry guidelines and standards.