Confiance.ai Program Software Engineering for a Trustworthy AI

Artificial Intelligence becomes more and more necessary to manage the complexity of the situations that human beings and automated systems must deal with. Academic research proposes continuously improvements to the original fully connected neural network architecture. But the performances of the new algorithms come with weaknesses that lead to famous failures of AI systems: incomprehensible crashes of autonomous cars, wrong decisions of classification systems and strange confidence score of prediction algorithms. These unexpected behaviors of AI make difficult the use of this new technology within critical systems. Today, the origins of these misbehaviors are generally known. Most of the time it is due to a lack of robustness of selected models or to bias or incompleteness in training data set. In the frame of its national strategy for AI, the French Government has launched the Confiance.ai program in October 2021. 9 big companies (Renault, Valeo, Safran, Airbus, Naval Group, Air Liquide, Atos, Sopra Steria, Thales) together with 4 academic institutions (CEA, INRIA, IRT St Exupéry, IRT SystemX) are building a development environment for trustworthy AI systems. This environment will propose methods and tools to guarantee the good behavior of AI based critical systems. Confiance.ai relies on concrete use cases brought by the industrial partners. Technical solutions exist to answer these use cases but there was no way to guarantee their performances to fit the expectations of quality or safety specialists. The output of Confiance.ai will propose a tooled methodology to convince them.