Designing Safe SuperIntelligence
摘要
Researchers face at least six challenges in developing safe, human-aligned superintelligence (SI). First, we need safe SI by design. Second, we need a transparent and understandable SI. Third, we need to maintain some level of control as SI outstrips human ability to monitor its behavior. Fourth, we need means to align SI initially and maintain alignment as SI increases in intelligence. Fifth, we need scalable safety mechanisms. Sixth, the design for SI must handle potential exponential changes in the SI’s level of intelligence. The current approach to using machine learning to develop opaque models, supplemented by RLHF to test in safety, cannot meet these challenges. We need a new approach emphasizing safety and alignment by design. This paper presents a novel design for SI, leveraging the collective intelligence of many human and AI agents using a rigorous, transparent architecture that supports problem-solving, learning, and self-improvement. The design is compatible with current LLMs and foundation models. It is less costly, more powerful, and faster to develop than training trillion-parameter LLMs. Most importantly, it maximizes alignment with broadly representative human values and maintains dynamic alignment even as the SI surpasses human monitoring capabilities.