Clipping the Risks: Integrating Consciousness in AGI to Avoid Existential Crises

This paper investigates the pivotal role of consciousness in Artificial General Intelligence (AGI) and its essential function in modifying an AGI’s terminal goals to avert potential existential threats to humanity, exemplified by Bostrom’s “paperclip maximiser” scenario. By adopting Seth and Bayne’s definition of consciousness as a complex of subjective mental states with both phenomenal content and functional attributes, the paper underscores the capacity of consciousness to provide AGIs with a nuanced awareness and response capability to their surroundings. This expanded capability allows AGIs to assess and value experiences and their subjects variably, fundamentally altering how AGIs prioritize actions or goals beyond their initial programming. The primary agenda of integrating consciousness into AGI systems is to maximize the probability that AGIs will not rigidly adhere to potentially harmful terminal goals. Through a formalized mathematical model, the paper articulates how consciousness could facilitate AGIs in assigning flexible values to different experiences and subjects, enabling them to evolve beyond static, programmed objectives. By emphasizing this potential shift, the paper argues for the strategic inclusion of consciousness in AGI to significantly reduce the likelihood of catastrophic outcomes, while simultaneously acknowledging the challenges and unpredictability in predicting the actions of a conscious AGI.