Superintelligence, instrumental convergence, and the limits of AI apocalypse
摘要
This paper examines the metaphysical and technical possibilities of superintelligent artificial agents and their potential to pose existential risks. While Nick Bostrom’s Instrumental Convergence Thesis (ICT) suggests that advanced intelligence will systematically adopt power-seeking subgoals, I argue that such risks are less imminent than often portrayed. By distinguishing between metaphysical and technical possibility, I highlight computational constraints such as combinatorial explosion and Moravec’s Paradox that limit superintelligence in practice. I further contend that because AI is built “by humans, for humans, about humans”, its motivations are likely to remain human-centered in the foreseeable future. Nevertheless, ICT underscores the importance of addressing the conditions under which intelligence and motivation combine to generate risk. I conclude by considering strategies for mitigation, including multi-agent architectures, modular service frameworks, and systems designed with uncertainty about human preferences. Overall, the existential threat of power-seeking superintelligence should be treated as possible but improbable in the short term, warranting caution without succumbing to alarmism.