A Framework for Managing Quality Requirements for Machine Learning-Based Software Systems

Systems containing Machine Learning (ML) are becoming common, and the tasks performed by such systems must meet certain quality thresholds, e.g., desired levels of transparency, safety, and trust. Recent research has identified challenges in defining and measuring the achievement of non-functional requirements (NFRs) for ML systems. Managing NFRs is particularly challenging due to the differing nature and definitions of NFRs for ML systems including non-deterministic behavior, the need to scope over different system components (e.g., data, models, and code), and difficulty in establishing new measurements (e.g., measuring explainability). To address these challenges, we propose a framework for identifying, prioritizing, specifying, and measuring attainment of NFRs for ML systems. We present a preliminary evaluation of the framework via an interview study with practitioners. The framework captures a first step towards enabling practitioners to systematically deliver high-quality ML systems.