Fundamentals of Numerical Optimization

This chapter summarizes fundamentals of numerical optimization. The material covered here is not supposed to be a systematic and exhaustive presentation of the subject, but it should give the reader sufficient background information necessary to understand the remaining parts of this book. We cover conventional methods, including gradient-based algorithms for both unconstrained and constrained optimization, derivative-free techniques such as pattern search, as well as global optimization by means of nature-inspired metaheuristics.