Estimation and Estimator Properties

Mathematical modeling is an important part of many fields of engineering and science. Mathematical models are used to simulate physical systems and test their behavior under different conditions. These mathematical models depend on model parameters that must be evaluated from measurements collected by scientists or engineers from physical systems operating under the conditions of validity of the mathematical model. For example, an aerospace engineer modeling a rocket collects measurements of position, velocity and acceleration that together with the equation of motion allow him to evaluate the parameters of the rocket’s model. An electronic engineer measures voltage and current at the input and output of an electronic device and uses his measurements to obtain an input-output model of the device. A wildlife biologist collects population data for species in a particular environment to develop a predator-prey model for wildlife management. The measurements in all the above applications include measurement errors that must be considered when evaluating the model parameters. The measurement errors can be deterministic, random, or both, depending on the application. The presence of errors in the measurements makes it necessary to collect far more measurements than the number of parameters to be evaluated because the measurement redundancy corrects for the errors in each measurement. The two most popular approaches for parameter estimation are the minimum square error or least-squares approach and the maximum likelihood approach. They are the subjects of later chapters. In this chapter we discuss properties of estimators that allow us to assess their quality. Because parameter estimates are based on noisy measurements, the estimates themselves are random. Even in cases where the noise distribution is known, the distribution of the parameter estimates can be quite complicated. However, to assess the quality of an estimate it is essential to learn more about its distribution and how its distribution is influenced by the size of the data sample used in estimation. In some cases, it is not possible to obtain this information for finite sample size, but much can be learned by taking the limit as the sample size goes to infinity. Properties evaluated by taking the limit are known as large sample properties. These properties allow us to evaluate the effect of increasing sample size on the quality of the estimator. Properties that can be evaluated without taking the limit are valid for any sample size and are known as small sample properties. We discuss small sample properties next.