Realizing the Promise of AI Governance Involving Humans-in-the-Loop

Human-in-the-loop (“HITL”) approaches have been proposed as an important element to ensure safety and fairness for higher-risk applications of artificial intelligence (“AI”) enabled decision-making. This paper examines the question: To what extent does the current state of knowledge enable the definition of factors and mitigations necessary for the effective implementation of HITL approaches? Previous research examining internal (individual) and external (implementation-related structural and contextual) factors in human decision-making are surveyed and assessed for potential relevance. This analysis highlights factors likely to be of relevance and reveals gaps which hinder the elaboration of factors and mitigations relevant to the effectiveness of HITL approaches. In order to realize the potential for HITL approaches to enhance safety for those reliant on or subject to AI systems, governance implementation will need to include minimum requirements applicable to both human overseers and to the context in which they work and are assessed. Factors identified as relevant include: complacency, persuasion, workload, conformity, unequal treatment of different kinds of error, and organizational polices and norms which may deter thorough review and human challenge of AI recommendations. There is also a need for further research: to enhance understanding of the practical significance of HITL effectiveness risks factors and their management; to enable the classification of HITL-implementation environments; and, to provide robust measurement methodologies to enable the identification of the most promising human candidates for HITL overseer roles.