Towards the Development of a Data Security Risk Management Framework for Medical Device Software AI Models

Data is considered the ‘backbone’ of the development of Artificial Intelligence (AI) models, including medical device software (MDS) AI models that process sensitive health data. Therefore, implementing necessary measures to assure data security is a key requirement that should be considered when developing MDS AI models. Developers face several challenges when assuring data security during the development of MDS AI models and the lack of guidance, i.e., a risk management standard or framework on managing the risks to sensitive health data is one of the major challenges they face. Moreover, the existing risk management standards and frameworks have several gaps and implementation challenges including: the lack of comprehensive threat and vulnerability lists; lack of a structured method for risk calculation or estimation; lack of a list of risk controls and risk control implementation details; and the need to refer to other standards and documentation for further information. Furthermore, current regulations and standards on AI model development recommend implementing a risk management process throughout the lifecycle of the AI model as a key requirement that should be employed for assuring data security. This paper presents the reasons behind the need for the development of a new developer friendly data security risk management framework that can be implemented by developers to assure data security when developing the MDS AI models. Additionally, this paper presents the elements that such a framework should contain. Ultimately, the framework should assist with improving the trustworthiness of AI and its adoption within the MDS industry and society.