Supervised Learning Paradigm
摘要
Supervised learning is one of the three major paradigms of machine learning. This chapter begins from the definition of supervised learning and explains its working principle using formal and illustrated descriptions. Second, we introduce the classic tasks of supervised learning. Next, we respectively discuss the two important factors to guarantee the performance for a supervised learning algorithm: one is the bias-variance problems including bias-variance trade-off and bias-variance decomposition, and another one is the risk minimization principles, namely expected risk minimization, empirical risk minimization, and structural risk minimization. We then introduce several variants of supervised learning, such as weakly supervised learning, semi-supervised learning, and label-free supervision, where the label-free supervision is a supervised learning approach based on domain knowledge without manual labeling. Finally, we introduce the no free lunch theorems, which are related to machine learning.