Canonical Optimization Algorithms

This chapter introduces canonical optimization algorithms, including numerical optimization methods for continuous optimization problems and state space search methods for discrete optimization problems. Several popular numerical optimization methods based on line search are presented, e.g., steepest descent method, Newton method, and conjugate gradient method. State space search methods are categorized into uninformed search methods and informed search methods. Uninformed search methods include breadth-first search, depth-first search, and depth-limited search. Informed search methods include greedy search, A* search, and Monte-Carlo tree search. Several single-solution-based metaheuristic search algorithms are also introduced in this chapter, e.g., hill climbing, simulated annealing, iterated local search, and variable neighborhood search.