Scholarlink

<p>Student’s <i>t</i>-test and Welch’s <i>t</i>-test are the most common defaults for comparing two independent samples, but each rests on often-violated assumptions. We compared the two tests in simulations varying sample-size imbalance, variances, and skewness, plus an empirical sampling study of gender differences on two psychological scales; further simulations compared both classical tests with the Yuen-Welch test, the <i>t</i>-test on ranks, Welch’s <i>t</i>-test on ranks, a permutation-based Welch’s test, the Brunner-Munzel test, the Kolmogorov–Smirnov test, and the Anderson–Darling test. Under unequal sample sizes, the <i>t</i>-test failed to maintain nominal Type I error when standard deviations also differed (the classical Welch motivation), while Welch’s test inflated Type I error when distributions were skewed, with the false positive rate reaching approximately 6%, 7.5%, and 9% at population skewness 1, 2, and 3 (nominal 5%). When sample sizes were equal, both classical tests held nominal Type I error; under unequal sample sizes with skewness, both lost power to robust alternatives, so the <i>t</i>-test was no remedy for Welch’s inflation. Among the alternatives, the permutation-based Welch’s test held the nominal Type I error across the factorial design while preserving the original measurement scale, making it a defensible default in the present simulations when the research question concerns equality of means. The Anderson–Darling test attained relatively high power when the two skewed populations differed simultaneously in mean and variability, and is a strong candidate when the research question concerns whether two distributions differ rather than whether their means differ specifically.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Neither the t-test nor Welch’s test: a case for robust alternatives in two-sample comparisons with non-ideal data

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月4日

查看全文
  • Joost de Winter

摘要

Student’s t-test and Welch’s t-test are the most common defaults for comparing two independent samples, but each rests on often-violated assumptions. We compared the two tests in simulations varying sample-size imbalance, variances, and skewness, plus an empirical sampling study of gender differences on two psychological scales; further simulations compared both classical tests with the Yuen-Welch test, the t-test on ranks, Welch’s t-test on ranks, a permutation-based Welch’s test, the Brunner-Munzel test, the Kolmogorov–Smirnov test, and the Anderson–Darling test. Under unequal sample sizes, the t-test failed to maintain nominal Type I error when standard deviations also differed (the classical Welch motivation), while Welch’s test inflated Type I error when distributions were skewed, with the false positive rate reaching approximately 6%, 7.5%, and 9% at population skewness 1, 2, and 3 (nominal 5%). When sample sizes were equal, both classical tests held nominal Type I error; under unequal sample sizes with skewness, both lost power to robust alternatives, so the t-test was no remedy for Welch’s inflation. Among the alternatives, the permutation-based Welch’s test held the nominal Type I error across the factorial design while preserving the original measurement scale, making it a defensible default in the present simulations when the research question concerns equality of means. The Anderson–Darling test attained relatively high power when the two skewed populations differed simultaneously in mean and variability, and is a strong candidate when the research question concerns whether two distributions differ rather than whether their means differ specifically.