Beginning Hypothesis Testing and Effect Size

This chapter begins the text’s instruction on inferential statistics. We use this chapter to describe the theoretical framework of hypothesis testing and effect sizes to provide that conceptual background to students before we teach the details of various inferential statistical analyses, which begin with the Independent Samples t-Test in Chap. 5 . The present chapter describes how the goal of experimental research is to push the mean of the experimental group far enough away from the control group and into the region of rejection to provide sufficient evidence that the treatment had an effect. We use the example of handing out free water bottles on campus in an attempt to change the hydration habits of our sample of college students compared to the college population average. In this example, our independent variable (giving out water bottles) is manipulated in order to try to push our sample mean away from the mean of the untreated population. We discuss Type I and Type II errors in the context of this example and cite relevant examples of “real-world” Type I errors: Bem’s research on precognition and the Reproducibility Project. We introduce the concept of effect size as a measure of treatment effect, emphasizing that many scientific journals now require estimates of effect size. Students will learn several different measures of effect size, such as Cohen’s d, r2, and η2, in future chapters. We discuss alpha and hypothesis testing to enable students to understand previously published research, which emphasized hypothesis testing. Illustrations of population distributions help students to understand what we mean by “representative” and “unrepresentative” samples as well as the region of rejection that is determined by alpha. One-tailed and two-tailed hypothesis tests are discussed. Students learn about the file drawer effect and p-hacking as issues that are central to the controversy of relying on p-values when we interpret statistical data. Biography boxes highlight Gertrude Cox and Jerzy Neyman. The chapter concludes with a chapter summary, a list of critical thinking questions, and practice questions.