Scholarlink

<p>Granting artificial intelligences (AIs) moral status creates a set of ethical dilemmas stemming from our ability to control their preferences. As an illustrative example, if we could program an artificial system worthy of moral agency to experience suffering when encountering the color violet, then we appear to have created a scenario where we have a moral imperative to remove any violet objects from our surroundings. AIs in this situation can effectively “hijack” our moral landscape, altering the moral valence of arbitrary world states. In this work, we formalize the phenomenon of moral hijacking, analyze its implications from the perspective of several ethical systems, and propose recommendations for constraining novel preference creation in moral agents.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A world without violet: peculiar consequences of granting moral status to Artificial Intelligences

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月19日

查看全文
  • Sever Ioan Topan

摘要

Granting artificial intelligences (AIs) moral status creates a set of ethical dilemmas stemming from our ability to control their preferences. As an illustrative example, if we could program an artificial system worthy of moral agency to experience suffering when encountering the color violet, then we appear to have created a scenario where we have a moral imperative to remove any violet objects from our surroundings. AIs in this situation can effectively “hijack” our moral landscape, altering the moral valence of arbitrary world states. In this work, we formalize the phenomenon of moral hijacking, analyze its implications from the perspective of several ethical systems, and propose recommendations for constraining novel preference creation in moral agents.