Ignoring criticisms of meta-analyses and systematic reviews is a risky strategy
摘要
Over the last two decades, concern has arisen about the potential of meta-analyses and systematic reviews, when they are incorrectly implemented, to yield erroneous conclusions. Substantial efforts have been devoted to ensure that these literature synthesis efforts are carried out and reported on rigorously, and that their conclusions are reliable. It appears that researchers in plant and soil sciences have been very slow to adopt “best practices” in these respects.
ScopeIn this opinion paper, I try to analyse in exactly what ways the criticisms of meta-analyses and systematic reviews that have emerged over the years have been virtually ignored in the literature on plant and soil sciences, and I argue that in the long run, this is not a very good strategy, because erroneous meta-analyses may mislead policy-makers and the public in ways that will eventually jeopardize the credibility of our disciplines.
ConclusionsEvidence suggests that criticisms of the meta-analytic methodology are ignored in 4 fundamental ways: 1) slow adoption of implementation and reporting guidelines, 2) disregard of criticisms of quality-control measures, 3) indiscriminate citations to problematic meta-analyses, and 4) lack of cautionary statements accompanying published meta-analyses, alerting readers of potential downfalls later on, as more information becomes available. All 4 of these practices, I argue, pose a severe risk because they allow the publication of meta-analyses that do not meet appropriate quality standards, and are therefore potentially misleading in the very long run. Several practical solutions are proposed, which would require action not only from researchers and journal editors but also from publishers, and would ensure some quality control in the meta-analyses that are published in the future.