Optimization Algorithms

This chapter mainly describes the fundamental concepts of some optimization algorithms. The gradient-based algorithms capable of updating the parameters based on the objective function’s derivative information are first introduced. Then, the least squares algorithm and its recursive version by minimizing the sum of squared errors are described. Another type of parameter optimization method by minimizing the variance estimation of the parameters, which is extended Kalman filter is introduced herein. Finally, a novel and fast learning algorithm named extreme learning machine is presented in the chapter. These constitute the research basis for the subsequent development of some novel sequential fuzzy systems.