Namespaces
Variants
Actions

Difference between revisions of "Likelihood-ratio test"

From Encyclopedia of Mathematics
Jump to: navigation, search
(Importing text file)
 
m (tex encoded by computer)
 
Line 1: Line 1:
A statistical test based on the ratio of the greatest values of the likelihood functions under the hypothesis being tested and under all possible states of nature. Let a random variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588101.png" /> have values in the sample space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588102.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588103.png" />, let the family of measures <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588104.png" /> be absolutely continuous with respect to a <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588105.png" />-finite measure <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588106.png" /> and let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588107.png" />. Suppose it is necessary, via a realization of the random variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588108.png" />, to test the composite hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l0588109.png" /> according to which the unknown true value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881010.png" /> of the parameter <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881011.png" /> belongs to the set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881012.png" />, against the composite alternative <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881013.png" />. According to the likelihood-ratio test with significance level <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881014.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881015.png" />, the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881016.png" /> has to be rejected if as a result of the experiment it turns out that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881017.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881018.png" /> is the statistic of the likelihood-ratio test, defined by:
+
<!--
 +
l0588101.png
 +
$#A+1 = 35 n = 0
 +
$#C+1 = 35 : ~/encyclopedia/old_files/data/L058/L.0508810 Likelihood\AAhratio test
 +
Automatically converted into TeX, above some diagnostics.
 +
Please remove this comment and the {{TEX|auto}} line below,
 +
if TeX found to be correct.
 +
-->
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881019.png" /></td> </tr></table>
+
{{TEX|auto}}
 +
{{TEX|done}}
  
while <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881020.png" /> is the critical level determined by the condition that the size of the test,
+
A statistical test based on the ratio of the greatest values of the likelihood functions under the hypothesis being tested and under all possible states of nature. Let a random variable  $  X $
 +
have values in the sample space  $  \{ \mathfrak X , {\mathcal B} , {\mathsf P} _  \theta  \} $,
 +
$  \theta \in \Theta $,
 +
let the family of measures  $  {\mathcal P} = \{ { {\mathsf P} _  \theta  } : {\theta \in \Theta } \} $
 +
be absolutely continuous with respect to a  $  \sigma $-
 +
finite measure  $  \mu $
 +
and let  $  p _  \theta  ( x) = d {\mathsf P} _  \theta  ( x)/d \mu ( x) $.
 +
Suppose it is necessary, via a realization of the random variable  $  X $,
 +
to test the composite hypothesis  $  H _ {0} $
 +
according to which the unknown true value  $  \theta _ {0} $
 +
of the parameter  $  \theta $
 +
belongs to the set  $  \Theta _ {0} \subset  \Theta $,
 +
against the composite alternative  $  H _ {1} :  \theta _ {0} \in \Theta _ {1} = \Theta \setminus  \Theta _ {0} $.  
 +
According to the likelihood-ratio test with significance level  $  \alpha $,
 +
$  0 < \alpha < 1/2 $,
 +
the hypothesis  $  H _ {0} $
 +
has to be rejected if as a result of the experiment it turns out that $  \lambda ( x) \leq  \lambda _  \alpha  $,
 +
where  $  \lambda ( X) $
 +
is the statistic of the likelihood-ratio test, defined by:
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881021.png" /></td> </tr></table>
+
$$
 +
\lambda ( X)  =
 +
\frac{\sup _ {\theta \in \Theta _ {0} } \
 +
p _  \theta  ( X) }{\sup _ {\theta \in \Theta }  p _  \theta  ( X) }
 +
,
 +
$$
  
is equal to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881022.png" />. In particular, if the set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881023.png" /> contains only two points <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881024.png" />, with densities <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881025.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881026.png" /> respectively, corresponding to the concurrent hypotheses which, in this case, are simple, then the statistic of the likelihood-ratio test is simply
+
while  $  \lambda _  \alpha  $
 +
is the critical level determined by the condition that the size of the test,
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881027.png" /></td> </tr></table>
+
$$
 +
\sup _ {\theta \in \Theta _ {0} }  {\mathsf P} _  \theta  \{ \lambda ( x) \leq  \lambda _  \alpha  \}  = \
 +
\sup _ {\theta \in \Theta _ {0} } \
 +
\int\limits _ {\{ {x } : {\lambda ( x) \leq  \lambda _  \alpha  } \}
 +
}
 +
p _  \theta  ( x) \mu ( dx) ,
 +
$$
  
According to the likelihood-ratio test with significance level <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881028.png" />, the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881029.png" /> has to be rejected if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881030.png" />, where the number <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881031.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881032.png" />, is determined by the condition
+
is equal to $  \alpha $.  
 +
In particular, if the set  $  \Theta $
 +
contains only two points  $  \Theta = \{ {\mathsf P} _ {0} , {\mathsf P} _ {1} \} $,
 +
with densities  $  p _ {0} ( \cdot ) $
 +
and  $  p _ {1} ( \cdot ) $
 +
respectively, corresponding to the concurrent hypotheses which, in this case, are simple, then the statistic of the likelihood-ratio test is simply
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881033.png" /></td> </tr></table>
+
$$
 +
\lambda ( X)  = \
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881034.png" /></td> </tr></table>
+
\frac{p _ {0} ( X) }{\max \{ p _ {0} ( X), p _ {1} ( X) \} }
  
The (generalized) likelihood-ratio test was proposed by J. Neyman and E.S. Pearson in 1928. They also proved (1933) that of all level-<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l058/l058810/l05881035.png" /> tests for testing one simple hypothesis against another, the likelihood-ratio test is the most powerful (see [[Neyman–Pearson lemma|Neyman–Pearson lemma]]).
+
= \
 +
\min \left \{ 1,
 +
\frac{p _ {0} ( X) }{p _ {1} ( X) }
 +
\right \} .
 +
$$
 +
 
 +
According to the likelihood-ratio test with significance level  $  \alpha $,
 +
the hypothesis  $  H _ {0} $
 +
has to be rejected if  $  p _ {0} ( X)/p _ {1} ( X) \leq  \lambda _  \alpha  $,
 +
where the number  $  \lambda _  \alpha  $,
 +
$  0 < \lambda _  \alpha  < 1 $,
 +
is determined by the condition
 +
 
 +
$$
 +
{\mathsf P} \{ \lambda ( X) < \lambda _  \alpha  \mid  H _ {0} \} =
 +
$$
 +
 
 +
$$
 +
= \
 +
\int\limits _ {\{ x:  p _ {0} ( x) \leq  p _ {1} ( x)
 +
\lambda _  \alpha  \} } p _ {0} ( x) \mu ( dx)  =  \alpha .
 +
$$
 +
 
 +
The (generalized) likelihood-ratio test was proposed by J. Neyman and E.S. Pearson in 1928. They also proved (1933) that of all level- $  \alpha $
 +
tests for testing one simple hypothesis against another, the likelihood-ratio test is the most powerful (see [[Neyman–Pearson lemma|Neyman–Pearson lemma]]).
  
 
====References====
 
====References====
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  J. Neyman,  E.S. Pearson,  "Joint statistical papers" , Cambridge Univ. Press  (1967)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  E.L. Lehmann,  "Testing statistical hypotheses" , Wiley  (1986)</TD></TR></table>
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  J. Neyman,  E.S. Pearson,  "Joint statistical papers" , Cambridge Univ. Press  (1967)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  E.L. Lehmann,  "Testing statistical hypotheses" , Wiley  (1986)</TD></TR></table>
 
 
  
 
====Comments====
 
====Comments====
 
This test is also called the generalized likelihood-ratio test, or the Wald test.
 
This test is also called the generalized likelihood-ratio test, or the Wald test.

Latest revision as of 22:16, 5 June 2020


A statistical test based on the ratio of the greatest values of the likelihood functions under the hypothesis being tested and under all possible states of nature. Let a random variable $ X $ have values in the sample space $ \{ \mathfrak X , {\mathcal B} , {\mathsf P} _ \theta \} $, $ \theta \in \Theta $, let the family of measures $ {\mathcal P} = \{ { {\mathsf P} _ \theta } : {\theta \in \Theta } \} $ be absolutely continuous with respect to a $ \sigma $- finite measure $ \mu $ and let $ p _ \theta ( x) = d {\mathsf P} _ \theta ( x)/d \mu ( x) $. Suppose it is necessary, via a realization of the random variable $ X $, to test the composite hypothesis $ H _ {0} $ according to which the unknown true value $ \theta _ {0} $ of the parameter $ \theta $ belongs to the set $ \Theta _ {0} \subset \Theta $, against the composite alternative $ H _ {1} : \theta _ {0} \in \Theta _ {1} = \Theta \setminus \Theta _ {0} $. According to the likelihood-ratio test with significance level $ \alpha $, $ 0 < \alpha < 1/2 $, the hypothesis $ H _ {0} $ has to be rejected if as a result of the experiment it turns out that $ \lambda ( x) \leq \lambda _ \alpha $, where $ \lambda ( X) $ is the statistic of the likelihood-ratio test, defined by:

$$ \lambda ( X) = \frac{\sup _ {\theta \in \Theta _ {0} } \ p _ \theta ( X) }{\sup _ {\theta \in \Theta } p _ \theta ( X) } , $$

while $ \lambda _ \alpha $ is the critical level determined by the condition that the size of the test,

$$ \sup _ {\theta \in \Theta _ {0} } {\mathsf P} _ \theta \{ \lambda ( x) \leq \lambda _ \alpha \} = \ \sup _ {\theta \in \Theta _ {0} } \ \int\limits _ {\{ {x } : {\lambda ( x) \leq \lambda _ \alpha } \} } p _ \theta ( x) \mu ( dx) , $$

is equal to $ \alpha $. In particular, if the set $ \Theta $ contains only two points $ \Theta = \{ {\mathsf P} _ {0} , {\mathsf P} _ {1} \} $, with densities $ p _ {0} ( \cdot ) $ and $ p _ {1} ( \cdot ) $ respectively, corresponding to the concurrent hypotheses which, in this case, are simple, then the statistic of the likelihood-ratio test is simply

$$ \lambda ( X) = \ \frac{p _ {0} ( X) }{\max \{ p _ {0} ( X), p _ {1} ( X) \} } = \ \min \left \{ 1, \frac{p _ {0} ( X) }{p _ {1} ( X) } \right \} . $$

According to the likelihood-ratio test with significance level $ \alpha $, the hypothesis $ H _ {0} $ has to be rejected if $ p _ {0} ( X)/p _ {1} ( X) \leq \lambda _ \alpha $, where the number $ \lambda _ \alpha $, $ 0 < \lambda _ \alpha < 1 $, is determined by the condition

$$ {\mathsf P} \{ \lambda ( X) < \lambda _ \alpha \mid H _ {0} \} = $$

$$ = \ \int\limits _ {\{ x: p _ {0} ( x) \leq p _ {1} ( x) \lambda _ \alpha \} } p _ {0} ( x) \mu ( dx) = \alpha . $$

The (generalized) likelihood-ratio test was proposed by J. Neyman and E.S. Pearson in 1928. They also proved (1933) that of all level- $ \alpha $ tests for testing one simple hypothesis against another, the likelihood-ratio test is the most powerful (see Neyman–Pearson lemma).

References

[1] J. Neyman, E.S. Pearson, "Joint statistical papers" , Cambridge Univ. Press (1967)
[2] E.L. Lehmann, "Testing statistical hypotheses" , Wiley (1986)

Comments

This test is also called the generalized likelihood-ratio test, or the Wald test.

How to Cite This Entry:
Likelihood-ratio test. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Likelihood-ratio_test&oldid=47635
This article was adapted from an original article by M.S. Nikulin (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article