Addressing Safety in AI-Based Systems: Insights from Systems Engineering

The rise of artificial intelligence (AI) has brought many changes to different domains like healthcare and autonomous vehicles. Yet, with its growth comes a daunting array of safety challenges. How can we ensure that an AI-based system is not just efficient but also robust against unexpected adversities? How can we trust its consistent performance over time? What mechanisms are in place to enable it to swiftly recover from disruptions? Moreover, in an ever-evolving digital landscape, can AI evolve stronger from the very disruptions that attempt to derail it? The paper highlights that the conventional safety paradigms and traditional approach of systems engineering (SE) may not be suitable for the complex nature of AI. Through a rigorous combination of literature review, methodological study, and comparative analysis, it critically evaluates the inadequacy of these conventional frameworks in addressing AI’s complexities. Due to its unpredictable behaviors and opaque decision-making processes, ensuring AI safety is a challenging task. A more rigorous methodology is required that leverages robustness, reliability, resilience, and antifragility principles to protect against known and unforeseen threats and ensure optimal performance as AI continues to be used across various industries.