Nonlinear Equations and Optimization

This chapter presents numerical methods used to locate the roots of systems of nonlinear equations or find the extrema of real-valued functions defined on a subset of Euclidean n-space. It demonstrates the close link between the two kinds of problem, since the solution of a root-finding problem can be transformed into a nonlinear minimization problem. With one exception (genetic search), the authors consider algorithms that are iterative, proceeding from a given initial point in successive steps. First, they develop stopping criteria for the iterative algorithms. Next, they consider standard methods used to solve nonlinear equations including the (modified) Newton-Raphson and Gauss-Seidel methods, before describing numerical optimization methods such as the golden section search, Newton’s method, and quasi Newton methods.