Algorithmic Stability
摘要
Recall the classical uniform generalization bounds, which essentially depend on a measure of the capacity of the hypothesis space \(\mathcal {H}\) (e.g., the Rademacher complexity or covering number). This type of upper bound is independent of any specific algorithm because it provides a guarantee for all hypotheses \(h \in \mathcal {H}\) simultaneously. In contrast to the above approaches, generalization bounds associated with a specific learning algorithm can be established based on the concept of algorithmic stability, which refers to the property that the hypothesis output by an algorithm does not change significantly when one training point is perturbed. In this chapter, we introduce algorithmic stability to characterize the impact of randomness in the training process on the generalizability of an algorithm.