A Selective Conceptual Review of CRISP-DM and DDSL Development Methodologies for Big Data Analytics Systems

Big data analytics systems (BDAS) have emerged through the convergence of analytics techniques and the availability of sources of massive data, internal and external to the organization, with descriptive, predictive, or prescriptive purposes. BDAS are relevant software systems pursued in diverse domains of application such as marketing, healthcare, finance, manufacturing, logistics, education, and tourism, among others. However, despite BDAS being a modern software system, its development has been conducted mainly using either ad hoc practical guidelines or old rigor-oriented heavyweight methodologies. The business competitive environment demands currently modern – i.e., lightweight or agile – BDAS development methodologies. However, despite some new BDAS development methodologies that have been proposed, studies contrasting rigor-oriented vs. lightweight BDAS development methodologies are still scarce in the literature. In this chapter, we address this knowledge gap, and using the ISO/IEC 29110 standard – Basic profile – as a theoretical expected lightweight development process, we report a selective conceptual review between CRISP-DM – the main rigor-oriented BDAS methodology – and DDSL, a new relevant proprietary lightweight one. Our selective comparative review provides theoretical and practical insights for discriminating both BDAS development approaches useful for researchers and practitioners in the domain of BDAS development projects.