Variable Selection

Effective variable selection plays a pivotal role in statistical modeling. We are frequently not only interested in using a model for prediction, but also need to correctly identify the relevant variables, i.e., to recover the correct model under given assumptions. It is known that under certain conditions, the ordinary least squares (OLS) method produces poor prediction results and does not yield a parsimonious model, resulting in overfitting. The objective of variable selection methods is to find the variables which are the most relevant ones for prediction. Such methods are particularly valuable, when the true underlying model has a sparse representation where many parameters are close to zero. The identification of relevant variables reduces noise and therefore improves the predictive performance of the model.