Affine Linear, Polynomial and Logistic Regression

The classical methods of simple, multivariable, and multivariate (affine) linear regression, polynomial regression, and logistic regression are discussed in detail. In the case of affine linear regression, we first consider the method of least squares from an optimization perspective. Then we adopt a probabilistic viewpoint and show that the affine linear regressor also maximizes a canonical likelihood function. Concerning logistic regression, we start directly with a maximum likelihood approach and then provide a detailed proof of the existence of the logistic regressor for overlapping datasets. We also discuss a sufficient condition for its uniqueness.