Maximum-likelihood method

One of the fundamental general methods for constructing estimators of unknown parameters in statistical estimation theory.

Suppose one has, for an observation with distribution depending on an unknown parameter , the task to estimate . Assuming that all measures are absolutely continuous relative to a common measure , the likelihood function is defined by

The maximum-likelihood method recommends taking as an estimator for the statistic defined by

is called the maximum-likelihood estimator. In a broad class of cases the maximum-likelihood estimator is the solution of a likelihood equation

(1)

Example 1. Let be a sequence of independent random variables (observations) with common distribution , . If there is a density

relative to some measure , then

and the equations (1) take the form

(2)

Example 2. In Example 1, let be the normal distribution with density

where , , , . Equations (2) become

and the maximum-likelihood estimator is given by

Example 3. In Example 1, let take the values and with probabilities , , respectively. Then

and the maximum-likelihood estimator is .

Example 4. Let the observation be a diffusion process with stochastic differential

where is a Wiener process and is an unknown one-dimensional parameter. Here (see [3]),

There are no definitive reasons for optimality of the maximum-likelihood method and the widespread belief in its efficiency is partially based on the great success with which it has been applied to numerous concrete problems, and partially on rigorously established asymptotic optimality properties. For example, in Example 1, under broad assumptions, with -probability . If the Fisher information

exists, then the difference is asymptotically normal with parameters , and , in a well-defined sense, has an asymptotically-minimal mean-square deviation from (see [4], [5]).

References

[1]	H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
[2]	S. Zacks, "The theory of statistical inference" , Wiley (1975)
[3]	R.S. Liptser, A.N. Shiryaev, "Statistics of random processes" , 1 , Springer (1977) (Translated from Russian)
[4]	A.I. Ibragimov, "Statistical estimation: asymptotic theory" , Springer (1981) (Translated from Russian)
[5]	E.L. Lehmann, "Theory of point estimation" , Wiley (1983)