Introduction

This chapter briefly introduces the background of optimization. Firstly, the relationship between optimization and machine learning is discussed, and an example of machine learning task is given. Secondly, the mathematical formulation of an optimization problem is defined. Optimization problems are categorized into continuous optimization problems and discrete optimization problems. Finally, optimization algorithms are introduced and categorized into deterministic algorithms and probabilistic algorithms, where several terms regarding intelligent optimization are introduced.