The debate around the Black Box problem crosses over a myriad of different disciplines, including the law. We analyze this debate and look at some of the ways that the problem has been approached in an effort to make AI more legible. We discuss how this Black Box problem lies at the heart of the use and abuse of AI and that it exists in a binary context, having both qualitative and quantitative aspects. We look at traditional tools for problem-solving, such as regulation, and how these do not fully address the qualitative or quantitative issues inherent in the Black Box problem, but also, in many ways, make the Black Box problem worse. In part, this is a result of the level of complexity that self-learners possess. In recent years, artificial intelligence models have grown in their complexity, evolving from simple, singular decision units to complex, interwoven systems. As the complexity of these models has grown, they have moved from displaying a simple, causality-driven operability, where one can look for what triggered the decision and find it. This is an effect that can be described as behavior resulting directly from a single action or cause. With today's AI, these triggers can raise their heads and exist in the system, but cannot easily be seen; in fact, they may have hidden triggers that we can’t label as a cause. There are correlations and causation frequently mixed up in complex settings. Such non-transparent environments lead to decisions in AI, despite the sophistication of today's methodology not being interpretable to the user interacting with the system; these might be internal agents or external agents. This results in a situation where decision-making is non-transparent, undermining human control in AI and hence posing considerable risks to the safety, operation, marketing, and branding of AI. It is these risks that we describe as the Black Box problem.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

The “Black Box” Problem: Lack of Transparency in AI Decision-Making

  • Wasswa Shafik

摘要

The debate around the Black Box problem crosses over a myriad of different disciplines, including the law. We analyze this debate and look at some of the ways that the problem has been approached in an effort to make AI more legible. We discuss how this Black Box problem lies at the heart of the use and abuse of AI and that it exists in a binary context, having both qualitative and quantitative aspects. We look at traditional tools for problem-solving, such as regulation, and how these do not fully address the qualitative or quantitative issues inherent in the Black Box problem, but also, in many ways, make the Black Box problem worse. In part, this is a result of the level of complexity that self-learners possess. In recent years, artificial intelligence models have grown in their complexity, evolving from simple, singular decision units to complex, interwoven systems. As the complexity of these models has grown, they have moved from displaying a simple, causality-driven operability, where one can look for what triggered the decision and find it. This is an effect that can be described as behavior resulting directly from a single action or cause. With today's AI, these triggers can raise their heads and exist in the system, but cannot easily be seen; in fact, they may have hidden triggers that we can’t label as a cause. There are correlations and causation frequently mixed up in complex settings. Such non-transparent environments lead to decisions in AI, despite the sophistication of today's methodology not being interpretable to the user interacting with the system; these might be internal agents or external agents. This results in a situation where decision-making is non-transparent, undermining human control in AI and hence posing considerable risks to the safety, operation, marketing, and branding of AI. It is these risks that we describe as the Black Box problem.