An Exploratory Big Data Approach to Understanding Commitment in Projects

This study addresses the twin challenges of talent retention and high project failure rates (40–70%) by harnessing machine learning (ML) techniques to analyze retrospective big data. The study’s objective was to ascertain whether project performance indicators can be a reliable gauge of project manager (PM) organizational commitment. This approach sidesteps the inherent bias and small effect sizes associated with survey self-report responses. Our innovative methodology leverages secondary big data, transforming the values into structured features that predict PM organizational commitment. This study proposes a novel conceptual framework, focusing on actual behavioral evidence rather than traditional, self-reported attitudes to assess the fuzzy predictors of organizational commitment. Among the three developed ML models, one demonstrated a significant 24% effect size, uncovering key features correlating PM tenure and organizational commitment with success. The insights gained from this research have broad implications for global stakeholders in projects and programs, offering a more objective and big data-driven understanding of PM commitment.