Repeatability and Reproducibility

Reproducibility is a cornerstone of the scientific process, as it allows researchers to independently verify and validate the findings of previous studies. Repeatability and reproducibility have already been discussed as a fundamental property of experiments in Chap. 4 . As per Definition 4.4 an experiment is repeatable when it shows the same outcomes under minimally different circumstances and an experiment is reproducible Definition 4.5 when it shows the same outcomes under specified circumstances. However, the fields of computer science and data science have been facing a reproducibility crisis, with many published results being difficult or impossible to reproduce (Baker, Nature 533(26):353–366, 2016). This crisis can be attributed to several factors, including the fact that the above definitions can, at first glance, seem subjective as it is hard to objectively define what “specified” or even “minimally different” circumstances mean.