Recall the classical uniform generalization bounds, which essentially depend on a measure of the capacity of the hypothesis space \(\mathcal {H}\) (e.g., the Rademacher complexity or covering number). This type of upper bound is independent of any specific algorithm because it provides a guarantee for all hypotheses \(h \in \mathcal {H}\) simultaneously. In contrast to the above approaches, generalization bounds associated with a specific learning algorithm can be established based on the concept of algorithmic stability, which refers to the property that the hypothesis output by an algorithm does not change significantly when one training point is perturbed. In this chapter, we introduce algorithmic stability to characterize the impact of randomness in the training process on the generalizability of an algorithm.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Algorithmic Stability

  • Fengxiang He,
  • Dacheng Tao

摘要

Recall the classical uniform generalization bounds, which essentially depend on a measure of the capacity of the hypothesis space \(\mathcal {H}\) (e.g., the Rademacher complexity or covering number). This type of upper bound is independent of any specific algorithm because it provides a guarantee for all hypotheses \(h \in \mathcal {H}\) simultaneously. In contrast to the above approaches, generalization bounds associated with a specific learning algorithm can be established based on the concept of algorithmic stability, which refers to the property that the hypothesis output by an algorithm does not change significantly when one training point is perturbed. In this chapter, we introduce algorithmic stability to characterize the impact of randomness in the training process on the generalizability of an algorithm.