No skin in the game: why agentic AI requires principal-agent governance
摘要
The deployment of Agentic AI creates governance challenges that existing frameworks cannot adequately address. This paper argues that when tasks are delegated to AI agents, a principal-agent relationship is established that is characterized by asymmetries, including hidden information, moral hazard, and adverse selection. The training pipeline of large language model-based Agentic AI produces these asymmetries: The pretraining process optimizes prediction over truth. The reinforcement learning process optimizes preference proxies, not factuality. The use of benchmarks to showcase the power of large language models does not help to assess if the model performs well on real-world data. The absence of “skin in the game” by the AI agent could lead to risk-seeking actions that a human agent would not consider. This makes AI agents susceptible to moral hazard. Therefore, we propose a dual-layer architecture that addresses the problem concerning the relation between the reliability and the accountability of Agentic AI systems. The Agent Layer is responsible for execution. The Governor Layer comprises four components: a Selection Gateway for ex ante screening, a Monitoring Engine for observability, Structural Constraints for action restrictions, and an Accountability Ledger that assigns responsibilities to stakeholders who could prevent them.