The probabilistic framework comes in handy when we need to quantify the uncertainty or stochastic processes in the observed data or the environment of machine learning. We in this chapter first take an overview on probability theory and probability learning theory. Reviewed the basic concepts and theorems of probability theory, we explain the theoretical foundation of machine learning, i.e., computational learning theory, that adopted the basic ideas of probability theory. Then we, respectively, introduce Bayesian models, Markov models, probabilistic models, probabilistic graphical models, and Monte Carlo methods, as they are all the important components in the probabilistic framework and the theoretical models for many machine learning algorithms.