Combating Overfitting
摘要
The flexibility of DNNs can be a blessing and a curse. While the crux of the optimization problem is to maximize performance on the training dataset, this is not actually our end goal. The true goal is for the model to perform well on the test set. While the distribution of the training data should closely resemble the data that we expect the model to encounter during deployment, they will not be identical. If we are not careful, the model may learn to exploit specific nuances of the training data, resulting in poor generalization on unseen samples. This phenomenon is known as overfitting. In this chapter, we discuss overfitting in detail and provide several methods for combating it.