Statistical Stockholm syndrome: the statistical significance myth is the problem
摘要
The idea that “statistical significance” somehow endows a result with special properties, revealing “real phenomena” vs. “results due only to chance,” is a myth. Therefore, “null results”—results found to be not “statistically significant”—should be treated like any other results, yielding a best parameter estimate and a measure of precision. The combination of parameter estimate and measure of precision does not tell the researcher anything beyond specifying a range within which the results of hypothetical identical replications of the current study are likely to fall. To make hypothesis test results useful for decision-making, researchers must incorporate —the expected costs of error and the cost of the research itself,—in contrast with the popular approach which ignores costs entirely. Incorporating fallacious “statistical significance” thinking into emerging artificial intelligence (AI) systems will produce new generations of wasteful research spending and misinterpreted results.