Regulatory Aspects and Ethical Legal Societal Implications (ELSI)

This chapter reviews the context of regulating AI/ML models, the risk management principles underlying international regulations of clinical AI/ML, the conditions under which health AI/ML models in the U.S. are regulated by the Food and Drug Administration (FDA), and the FDA’s Good Machine Learning Practice (GMLP) principles. The GMLP principles do not offer specific guidance on execution, so we point the Reader to the parts of the book that discuss bringing these principles to practice via concrete best practice recommendations. Intrinsically linked with regulatory aspects are the Ethical, Legal, Social Implications (ELSI) dimensions. The chapter provides an introduction to the nascent field of biomedical AI ethics covering: general AI ELSI studies, AI/ML racial bias, and AI/ML and Health equity principles. Contrary to conventional risks/harms (data security and privacy, adherence to model use as stated in consent), ethical AI/ML involves model effectiveness and harms that can exist within the intended scope of consent. On the positive side, in the case of biomedical AI, these risks are in principle measurable and knowable compared to hard-to-quantify risks/harm due to data breaches. The chapter discusses (and gives illustrative examples) of the importance of causality and equivalence classes for practical detection of racial bias in models. The chapter concludes with a series of recommended best practices for promoting health equity and reducing health disparities via the design and use of health AI/ML.