Principal Component Analysis

In this chapter we discuss some general techniques for statistical data compression (or noise reduction). These techniques can be used for the purpose of feature extraction in decision problems and have acquired a great importance in applications to classification. A couple of such significant applications will be briefly illustrated. There is also a large body of applications of the underlying compression idea to regression problems. The second part of the chapter could be described as “reduced-data” regression and goes under the name of Canonical Correlation Analysis which has a deep statistical significance. We analyze it both from a probabilistic perspective and from an algorithmic viewpoint.