As artificial intelligence (AI) systems become increasingly integrated into critical decision-making processes, ensuring their reliability, transparency, and safety is essential. This paper proposes a theoretical architecture for an AI Cockpit, a supervisory control interface designed to mitigate risks associated with AI applications. The AI Cockpit follows a hybrid intelligence approach, where AI systems and human operators collaborate as AI-Teams to optimize decision-making. This enables dynamic adjustments in levels of automation, maintaining a balance between automation efficiency and human oversight. We present an implementation of this architecture and demonstrate its applicability through three case studies. Our findings suggest that the proposed AI Cockpit architecture facilitates human-AI collaboration and improves risk management in AI-driven environments.