Artificial intelligence (AI) introduces unprecedented uncertainty in military operations. This is particularly evident in AI-enabled autonomous weapon systems (AWS) and decision support systems (DSS), which not only influence critical battlefield decisions but also pose novel and unpredictable risks. While some risks can be anticipated and managed ex ante, many remain inherent and unavoidable, given the complex, dynamic, and adversarial nature of the environments in which these systems operate. Even AI operators acting in good faith may face situations in which unforeseeable civilian harm occurs, despite rigorous review and careful deployment. In practice, many such incidents will be characterised as ‘accidents’—a reality of war that International Humanitarian Law is expected to tolerate. This chapter challenges that assumption, arguing that even a priori unpredictable AI failures can be mitigated—if not prevented—through an iterative approach. By systematically integrating insights from post-deployment assessments, this approach enables decision makers to update their understanding of edge cases and other ‘known unknowns’ that emerge during real-world use, providing essential insights to inform future AI deployment. It proposes an Iterative Assessment framework—implemented through two complementary mechanisms: Iterative Review and Iterative Assessment in Deployment. This framework represents best practice for managing uncertainty and minimising civilian harm in the use of military AI. While initial accidents may be unavoidable, their recurrence can be significantly reduced through a structured iterative process of reporting, analysis, and adaptation. Those committed to the responsible use of military AI should embed this framework as a core component of operational planning and legal compliance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Iterative Assessment for Military Artificial Intelligence (AI) Systems

  • Jonathan Kwik

摘要

Artificial intelligence (AI) introduces unprecedented uncertainty in military operations. This is particularly evident in AI-enabled autonomous weapon systems (AWS) and decision support systems (DSS), which not only influence critical battlefield decisions but also pose novel and unpredictable risks. While some risks can be anticipated and managed ex ante, many remain inherent and unavoidable, given the complex, dynamic, and adversarial nature of the environments in which these systems operate. Even AI operators acting in good faith may face situations in which unforeseeable civilian harm occurs, despite rigorous review and careful deployment. In practice, many such incidents will be characterised as ‘accidents’—a reality of war that International Humanitarian Law is expected to tolerate. This chapter challenges that assumption, arguing that even a priori unpredictable AI failures can be mitigated—if not prevented—through an iterative approach. By systematically integrating insights from post-deployment assessments, this approach enables decision makers to update their understanding of edge cases and other ‘known unknowns’ that emerge during real-world use, providing essential insights to inform future AI deployment. It proposes an Iterative Assessment framework—implemented through two complementary mechanisms: Iterative Review and Iterative Assessment in Deployment. This framework represents best practice for managing uncertainty and minimising civilian harm in the use of military AI. While initial accidents may be unavoidable, their recurrence can be significantly reduced through a structured iterative process of reporting, analysis, and adaptation. Those committed to the responsible use of military AI should embed this framework as a core component of operational planning and legal compliance.