Determining parameters of a model from data is often the most challenging part of the modeling process. The standard procedure for parameter estimation is nonlinear least squares (NLS) which produces maximum likelihood estimates (MLE). NLS is an optimization problem: it minimizes the square of the 2-norm of the residual. If the latter is differentiable, which is typically the case, the minimum can be found by setting the gradient to zero and solving the resulting system of algebraic equations. Here the weapon of choice is Newton’s method which is first introduced in one dimension and then generalized to multiple dimensions. When the data has outliers, the 2-norm in NLS may be replaced with the more robust 1-norm. However, the use of 1-norm makes the objective function non-differentiable which rules out Newton’s method and calls for general optimization algorithms, such as the simplex method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Estimation of Parameters

  • Aleksei Beltukov

摘要

Determining parameters of a model from data is often the most challenging part of the modeling process. The standard procedure for parameter estimation is nonlinear least squares (NLS) which produces maximum likelihood estimates (MLE). NLS is an optimization problem: it minimizes the square of the 2-norm of the residual. If the latter is differentiable, which is typically the case, the minimum can be found by setting the gradient to zero and solving the resulting system of algebraic equations. Here the weapon of choice is Newton’s method which is first introduced in one dimension and then generalized to multiple dimensions. When the data has outliers, the 2-norm in NLS may be replaced with the more robust 1-norm. However, the use of 1-norm makes the objective function non-differentiable which rules out Newton’s method and calls for general optimization algorithms, such as the simplex method.