Estimation of Parameters
摘要
Determining parameters of a model from data is often the most challenging part of the modeling process. The standard procedure for parameter estimation is nonlinear least squares (NLS) which produces maximum likelihood estimates (MLE). NLS is an optimization problem: it minimizes the square of the 2-norm of the residual. If the latter is differentiable, which is typically the case, the minimum can be found by setting the gradient to zero and solving the resulting system of algebraic equations. Here the weapon of choice is Newton’s method which is first introduced in one dimension and then generalized to multiple dimensions. When the data has outliers, the 2-norm in NLS may be replaced with the more robust 1-norm. However, the use of 1-norm makes the objective function non-differentiable which rules out Newton’s method and calls for general optimization algorithms, such as the simplex method.