Requirements for Machine Learning Methodology Software Tooling

A number of machine learning process models (SEMMA, KDD, CRISP-DM, CRISP-ML(Q), Data-to-Value, etc.) have been recently proposed to facilitate the development of machine learning models in their organizational context. While the existing proposals vary with respect to complexity and suitability for particular tasks, it would be desirable to have software tools that embody and support these methodologies and make it easier for project teams to capture, share among team members and stakeholders, and preserve the relevant project information pertaining to the various process stages. Various existing software systems cover parts such as team and communication management (Confluence, Jira, Slack, Zoom, etc.), project management (scrum, kanban, etc.), data and information management (Model Management Platform, cf. (Weber and Hirmer, Business Information Systems. Springer International Publishing, Cham, 2020), inter alia), or experimentation (RapidMiner, Orange, Weka, Tensorflow, etc.), but we are not aware of any management tools that tie them together and ensure methodology compliance. To the best of our knowledge, to date, no requirement analysis exists for a system that meets the need to provide guidance to teams for how to follow a machine learning methodology nor for managing all of a project’s metadata throughout its entire life cycle. To this end, we present an analysis and resulting collection of a set of 29 requirements for the software tooling for machine learning methodologies, derived from properties of the methodologies, user stories, and introspection of the authors.