Non-parametric Methods for Regression

Parametric approaches for regression, discussed in the previous chapter, requires a priori knowledge of the functional form to be fitted. This inherently assumes that the nature of the dataset represented by the distribution and the respective central and higher order measures is known. In cases where the nature of the data is not known, non-parametric methods are highly useful. Non-parametric methods are not limited by the distribution of the data and hence can be used for any dataset. In this chapter, we focus on non-parametric methods for regression. First, we discuss about tree-based approaches, such as regression treeRegression tree, random forestRandom forest, and gradient boosted trees. Then we discuss about multi-layer perceptron, popularly known as neural network and support vector regression. Finally, we discuss about a non-parametric probabilistic approach for regression, namely, Gaussian process regression.