<p>As artificial intelligence systems acquire increasing autonomy, the question of how humans can maintain meaningful control over their behavior becomes urgent. This question is especially pressing for agentic AI: systems built on foundation models that coordinate networks of autonomous agents to pursue complex goals with minimal human oversight. While the engineering literature has developed several mechanisms for constraining such systems, it has largely treated controllability as a purely causal property, the ability to physically intervene in a system’s operation, without engaging the normative dimensions that determine whether such intervention is meaningful. Drawing on the distinction between causal and normative control in the philosophy of action, this paper presents a structured conceptual survey of controllability mechanisms in agentic AI. We organize the literature around four paradigms: constraints and guardrails, adaptive control via reinforcement learning, agent-in-the-loop oversight, and human-in-the-loop oversight. For each, we analyze its technical foundations, its philosophical assumptions about agency and authority, and its persistent limitations. Our analysis reveals unresolved tensions across these paradigms, particularly between the scalability of automated oversight and the legitimacy of human judgment, between design-time constraint specification and runtime adaptability, and between individual agent control and system-level governance. We argue that meaningful controllability requires integrating causal intervention mechanisms with normative frameworks that address value alignment, epistemic transparency, and distributed responsibility. We conclude by proposing directions for research that bridges engineering controllability and philosophically grounded human oversight.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

On Controllability in Agentic AI: A Survey

  • My H. Nguyen,
  • Duc-Hai Nguyen,
  • Barry O’Sullivan,
  • Hoang D. Nguyen

摘要

As artificial intelligence systems acquire increasing autonomy, the question of how humans can maintain meaningful control over their behavior becomes urgent. This question is especially pressing for agentic AI: systems built on foundation models that coordinate networks of autonomous agents to pursue complex goals with minimal human oversight. While the engineering literature has developed several mechanisms for constraining such systems, it has largely treated controllability as a purely causal property, the ability to physically intervene in a system’s operation, without engaging the normative dimensions that determine whether such intervention is meaningful. Drawing on the distinction between causal and normative control in the philosophy of action, this paper presents a structured conceptual survey of controllability mechanisms in agentic AI. We organize the literature around four paradigms: constraints and guardrails, adaptive control via reinforcement learning, agent-in-the-loop oversight, and human-in-the-loop oversight. For each, we analyze its technical foundations, its philosophical assumptions about agency and authority, and its persistent limitations. Our analysis reveals unresolved tensions across these paradigms, particularly between the scalability of automated oversight and the legitimacy of human judgment, between design-time constraint specification and runtime adaptability, and between individual agent control and system-level governance. We argue that meaningful controllability requires integrating causal intervention mechanisms with normative frameworks that address value alignment, epistemic transparency, and distributed responsibility. We conclude by proposing directions for research that bridges engineering controllability and philosophically grounded human oversight.