Big Data Adoption Factors and Development Methodologies: A Multiple Case Study Analysis

Data is one of the most valuable resources in any organization. Big data primarily provides access to the data’s often untapped potential. In this research, there are two main focal areas. First, we strive to identify critical factors affecting the adoption of BD implementation in organizations using the interpretative phenomenological analysis (IPA) and technology-organization-environment (TOE) framework. Factors affecting BD adoption in this research include finding the appropriate use case to extract value, the challenge with security, the challenge of managing large datasets, privacy concerns, cost concerns, the burden of regulation, and the challenge of finding big data IT expertise. Second, we explored the organization’s BD development methodologies using IPA methodology to examine its successes and challenges. Most organizations examined in this research are using agile with medium-size teams. They have used agile development methodology that enabled them to create rapid development, continuous improvement, increased stakeholder participation, and ability to develop with incomplete big data expertise. It also has some challenges that include repeated conflict, feature interaction regressions, divergence of development paths, and longer development cycles due to experimentation.