Empirical Distribution Function

This short chapter is located at the interface between probability theory and mathematical statistics. You first learn about the empirical distribution function of n independent random variables, which are distributed according to the same unknown distribution function F. The main result of this chapter is the Glivenko-Cantelli theorem, which is also known under the naming fundamental theorem of statistics. According to this theorem, the empirical distribution function converges with probability one uniformly on the whole real line to F, as n tends to infinity. In statistical terms, the empirical distribution function is thus a strongly consistent estimator of F. The chapter also addresses higher-order aspects, such as the almost sure uniform convergence of empirical distributions, and the Dvoretsky-Kiefer-Wolfowitz inequality, which implies the Glivenko-Cantelli theorem.