<p>It is widely assumed that the principal reason to pursue or not pursue superintelligent AI (Super-AI) is the impact that such technology would have on the well-being of humans in general. I maintain, to the contrary, that the principal reason to pursue Super-AI is not for our sake, but for its own sake: all else being equal, if one world contains vastly more knowledge than another, then it is significantly better than the other; and a world in which Super-AI exists contains vastly more knowledge than a world in which it doesn’t. Accordingly, I maintain that, to the extent that we are morally obligated to make the world a better place, we are morally obligated to create Super-AI. Moreover, I argue that, because the knowledge that Super-AI would possess is so valuable, we ought to create it even if we knew in advance that doing so would precipitate human extinction: it would no more be reasonable to forgo the existence of Super-AIs in order to preserve the existence of human beings, than it would be to forgo the existence of human beings in order to preserve the existence of chimpanzees. Finally, I argue that, regardless of the threat of extinction, creating Super-AI is in our own best-interests. We all want to help ensure that there is much more knowledge in the future, even a future that we won’t live to see, than there is at present; and the best way to achieve this goal is to create Super-AI.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

The value of knowledge and the case for superintelligence

  • Boyd Millar

摘要

It is widely assumed that the principal reason to pursue or not pursue superintelligent AI (Super-AI) is the impact that such technology would have on the well-being of humans in general. I maintain, to the contrary, that the principal reason to pursue Super-AI is not for our sake, but for its own sake: all else being equal, if one world contains vastly more knowledge than another, then it is significantly better than the other; and a world in which Super-AI exists contains vastly more knowledge than a world in which it doesn’t. Accordingly, I maintain that, to the extent that we are morally obligated to make the world a better place, we are morally obligated to create Super-AI. Moreover, I argue that, because the knowledge that Super-AI would possess is so valuable, we ought to create it even if we knew in advance that doing so would precipitate human extinction: it would no more be reasonable to forgo the existence of Super-AIs in order to preserve the existence of human beings, than it would be to forgo the existence of human beings in order to preserve the existence of chimpanzees. Finally, I argue that, regardless of the threat of extinction, creating Super-AI is in our own best-interests. We all want to help ensure that there is much more knowledge in the future, even a future that we won’t live to see, than there is at present; and the best way to achieve this goal is to create Super-AI.