Methods of Testing a ‘Small’ or ‘Moderate’ Number of Hypotheses Simultaneously: An Account Focusing on the Control of the Probability of at Least One Incorrect Rejection and of the False Discovery Rate

This paper describes methods of multiple testing which we think are especially useful in applications involving a ‘small’ or ‘moderate’ number of hypotheses, say in the range of 5–50 rather than in the hundreds or thousands, for which the most interesting error rates appear to be the probability of at least one incorrect rejection and the false discovery rate. It is a review of mostly known methods, but one which presents proofs and justifications in an essentially self-contained and original manner. Although emphasis is given to methods that control error rates exactly, namely to those of Holm, Westfall and Young, and Benjamini and Hochberg, two methods designed to control the first of those error rates approximately, one based on the central limit theorem and the other on bootstrapping, are also considered.