Maximum-likelihood method
One of the fundamental general methods for constructing estimators of unknown parameters in statistical estimation theory.
Suppose one has, for an observation with distribution depending on an unknown parameter , the task to estimate . Assuming that all measures are absolutely continuous relative to a common measure , the likelihood function is defined by
The maximum-likelihood method recommends taking as an estimator for the statistic defined by
is called the maximum-likelihood estimator. In a broad class of cases the maximum-likelihood estimator is the solution of a likelihood equation
(1) |
Example 1. Let be a sequence of independent random variables (observations) with common distribution , . If there is a density
relative to some measure , then
and the equations (1) take the form
(2) |
Example 2. In Example 1, let be the normal distribution with density
where , , , . Equations (2) become
and the maximum-likelihood estimator is given by
Example 3. In Example 1, let take the values and with probabilities , , respectively. Then
and the maximum-likelihood estimator is .
Example 4. Let the observation be a diffusion process with stochastic differential
where is a Wiener process and is an unknown one-dimensional parameter. Here (see [3]),
There are no definitive reasons for optimality of the maximum-likelihood method and the widespread belief in its efficiency is partially based on the great success with which it has been applied to numerous concrete problems, and partially on rigorously established asymptotic optimality properties. For example, in Example 1, under broad assumptions, with -probability . If the Fisher information
exists, then the difference is asymptotically normal with parameters , and , in a well-defined sense, has an asymptotically-minimal mean-square deviation from (see [4], [5]).
References
[1] | H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946) |
[2] | S. Zacks, "The theory of statistical inference" , Wiley (1975) |
[3] | R.S. Liptser, A.N. Shiryaev, "Statistics of random processes" , 1 , Springer (1977) (Translated from Russian) |
[4] | A.I. Ibragimov, "Statistical estimation: asymptotic theory" , Springer (1981) (Translated from Russian) |
[5] | E.L. Lehmann, "Theory of point estimation" , Wiley (1983) |
Maximum-likelihood method. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Maximum-likelihood_method&oldid=13827