An Introduction of Statistical Learning for Engineers

Statistical learning is a branch of machine learning that is concerned with the statistical aspects of data analysis by means of machine learning algorithms. In the following, we present a leisurely introduction to this topic, as it is of particular interest for applications in engineering research and practice. Statistics is one of the roots of modern machine learning, in particular the discipline of nonparametric statistics, which more than 40 years ago developed a lot of concepts and ideas that are still relevant nowadays, e.g., tree-based learning (Sect. 7), local smoothing (Sect. 10), or resampling techniques like crossvalidation (Sect. 3.4) or bootstrap (Sect. 9.1). Using the advances of computational technology at that time, it was possible to leave the restricted framework of classical statistics, which mainly is based on models for the data-generating process, which are completely specified up to a manageable number of parameters. The latter could be estimated from the small- and medium-sized datasets that were typical in the past. However, researchers always were aware that those models only were an approximation to truth, in particular in realistic situations apart from ideal laboratory conditions, and that this model misspecification may be the cause of suboptimal or even false results from a data analysis. Nonparametric statistics allowed to adopt a much broader view on the data-generating process (compare Sect. 2.6), which is also the basis of modern machine learning methods.