Linear Discriminant Functions and Their Learning

In the previous chapter, we described a method for separating classes by means of a hyperplane designed with one prototype per class. In this case, it is important to determine the position of the prototype in the feature space. As shown in the previous chapter, a simple method that aligns the prototype with the center of gravity of the distribution does not always yield correct results. The learning procedure described in this chapter automatically finds the correct position of the prototypes in the high-dimensional feature space and divides the feature space into classes. This chapter first describes the relationship between the nearest neighbor rule and the linear discriminant function. Then, we explain the perceptron learning rule, a method for learning linear discriminant functions, with experiments. The piecewise linear discriminant function, which is an advanced form of the linear discriminant function, is also discussed. In addition, we discuss the concepts of margin, component vectors, and how they relate to support vector machines, which will be covered in Chap. 10 .