AGI, Existential Risks, and the Control Problem
摘要
This chapter explores the origins and evolution of concerns surrounding the long-term consequences of advanced AI development, particularly those linked to existential risks and the so-called “control problem”. At the same time, popular culture has long reflected anxieties about intelligent machines, and academic discussions of alignment and controllability were, until recently, marginalized. These concerns have become mainstream, fueled by technical breakthroughs and the recognition that aligned systems may have commercial value. This chapter will examine how early warnings—rooted in science fiction, philosophy, and early AI thought—laid the foundation for what is now formalized as AI safety and alignment research. By doing so, we offer a grounded introduction to the alignment problem as a central challenge in the future of AI development.