In the previous chapter, we described the learning algorithm in a more general and unified perspective in the framework of expected loss minimization and explained square error loss, zero-one loss, and continuous loss as loss functions. Furthermore, in the framework of minimum expected loss learning, we showed that the least squares method is derived for square error loss. In this chapter, learning based on the least squares method is described in detail. Specifically, we first derive the least squares solution of the decision function for the linear model and the nonlinear model. Next, the relationship between the least squares method and linear and nonlinear discriminant methods will be discussed. We also explain the relationship between the least squares method and the Bayes decision rule, and its relationship with Widrow–Hoff learning rule and the backpropagation method.