In this chapter, we look at the model selection problem in the supervised learning context. Starting with an introduction of the problem setting, we move to some empirical observations that demonstrate the prevalent tendencies of underfitting and overfitting in our trained models. This is followed by an inspection of error decomposition, identifying the roles of bias and variance in models of different complexities, and explaining the famous bias-variance dilemma.