Maximum-likelihood method
One of the fundamental general methods for constructing estimators of unknown parameters in statistical estimation theory.
Suppose one has, for an observation with distribution
depending on an unknown parameter
, the task to estimate
. Assuming that all measures
are absolutely continuous relative to a common measure
, the likelihood function is defined by
![]() |
The maximum-likelihood method recommends taking as an estimator for the statistic
defined by
![]() |
is called the maximum-likelihood estimator. In a broad class of cases the maximum-likelihood estimator is the solution of a likelihood equation
![]() | (1) |
Example 1. Let be a sequence of independent random variables (observations) with common distribution
,
. If there is a density
![]() |
relative to some measure , then
![]() |
and the equations (1) take the form
![]() | (2) |
Example 2. In Example 1, let be the normal distribution with density
![]() |
where ,
,
,
. Equations (2) become
![]() |
![]() |
and the maximum-likelihood estimator is given by
![]() |
Example 3. In Example 1, let take the values
and
with probabilities
,
, respectively. Then
![]() |
and the maximum-likelihood estimator is .
Example 4. Let the observation be a diffusion process with stochastic differential
![]() |
where is a Wiener process and
is an unknown one-dimensional parameter. Here (see [3]),
![]() |
![]() |
There are no definitive reasons for optimality of the maximum-likelihood method and the widespread belief in its efficiency is partially based on the great success with which it has been applied to numerous concrete problems, and partially on rigorously established asymptotic optimality properties. For example, in Example 1, under broad assumptions, with
-probability
. If the Fisher information
![]() |
exists, then the difference is asymptotically normal with parameters
, and
, in a well-defined sense, has an asymptotically-minimal mean-square deviation from
(see [4], [5]).
References
[1] | H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946) |
[2] | S. Zacks, "The theory of statistical inference" , Wiley (1975) |
[3] | R.S. Liptser, A.N. Shiryaev, "Statistics of random processes" , 1 , Springer (1977) (Translated from Russian) |
[4] | A.I. Ibragimov, "Statistical estimation: asymptotic theory" , Springer (1981) (Translated from Russian) |
[5] | E.L. Lehmann, "Theory of point estimation" , Wiley (1983) |
Maximum-likelihood method. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Maximum-likelihood_method&oldid=47805