Significance test

One of the basic methods for statistical hypotheses testing (cf. Statistical hypotheses, verification of), used to confront the outcomes of observations $ x _ {1} \dots x _ {n} $, interpreted as realizations of random variables $ X _ {1} \dots X _ {n} $, with a hypothesis $ H _ {0} $. It is usually based on the observed value of a suitable statistic $ T = T ( X _ {1} \dots X _ {n} ) $, the specific form of which depends on the formulation of the problem. Generally speaking, the application of a significance test does not presuppose the availability of an alternative hypothesis $ H _ {1} $, to be accepted if $ H _ {0} $ is rejected; however, if some $ H _ {1} $ is specified, then, according to the general theory of statistical hypotheses testing, it is precisely $ H _ {1} $ that determines the choice of the statistic $ T $, the governing principle being to maximize the power of the test (cf. Power of a statistical test).

Significance tests are usually applied as follows. Having selected the test by choosing in advance a statistic $ T ( X _ {1} \dots X _ {n} ) $ and a significance level $ \alpha $, $ 0 < \alpha < 0.5 $, determine the critical value $ t _ \alpha $ of the test by the condition $ P \{ T \geq t _ \alpha \mid H _ {0} \} = \alpha $. According to the significance test with level $ \alpha $, the hypothesis $ H _ {0} $ is rejected if $ T ( x _ {1} \dots x _ {n} ) \geq t _ \alpha $. If $ T ( x _ {1} \dots x _ {n} ) < t _ \alpha $, the conclusion is that $ H _ {0} $ is not in disagreement with the observations $ x _ {1} \dots x _ {n} $, at least as long as new observations do not oblige the experimenter to adopt a different point of view.

Example. A counter monitoring a Poisson process registers 150 pulses during the first hour, 117 pulses during the second. Can it be assumed on this basis that the pulse arrival rate per unit time is constant (hypothesis $ H _ {0} $)?

If $ H _ {0} $ is true, the observed values 150, 117 may be treated as realizations of two independent random variables $ X _ {1} $ and $ X _ {2} $ obeying the same Poisson law with parameter $ \lambda $; the value of $ \lambda $ is unknown. Since $ H _ {0} $ implies that the random variables

$$ Y _ {1} = \sqrt {4 X _ {1} + 1 } - 2 \sqrt \lambda \ \textrm{ and } \ Y _ {2} = \ \sqrt {4 X _ {2} + 1 } - 2 \sqrt \lambda $$

are approximately normally distributed with parameters $ ( 0 , 1) $, it follows that the statistic

$$ X ^ {2} = \frac{1}{2} [ Y _ {1} - Y _ {2} ] ^ {2} = \frac{1}{2} [ \sqrt {4 X _ {1} + 1 } - \sqrt {4 X _ {2} + 1 } ] ^ {2} $$

is approximately distributed according to a "chi-squared" law with one degree of freedom, i.e.

$$ {\mathsf P} \{ X ^ {2} > x \mid H _ {0} \} \approx {\mathsf P} \{ \chi _ {1} ^ {2} > x \} . $$

Tables of the "chi-squared" distribution yield the critical value $ \chi _ {1} ^ {2} ( 0.05 ) = 3.841 $ for the prescribed significance level $ \alpha = 0.05 $, i.e.

$$ {\mathsf P} \{ \chi _ {1} ^ {2} \geq 3.841 \} = 0.05 . $$

Now, using the observed values $ X _ {1} = 150 $ and $ X _ {2} = 117 $, one computes the value $ X ^ {2} $ of the test statistic:

$$ X ^ {2} = \frac{1}{2} [ \sqrt {4 \cdot 150 + 1 } - \sqrt {4 \cdot 117 + 1 } ] ^ {2} = 4.087 . $$

Since $ X ^ {2} = 4.085 > \chi _ {1} ^ {2} ( 0.05 ) = 3.841 $, it follows that the hypothesis $ H _ {0} $( constant pulse arrival rate) is rejected by the "chi-squared" test at significance level $ \alpha = 0.05 $.

References

[1]	H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
[2]	E.L. Lehmann, "Testing statistical hypotheses" , Wiley (1959)
[3]	N.V. Smirnov, I.V. Dunin-Barkovskii, "Mathematische Statistik in der Technik" , Deutsch. Verlag Wissenschaft. (1969) (Translated from Russian)
[4]	B.I. Devyatov, "Limits of admissibility of normal approximations to the Poisson distribution" Theor. Probab. Appl. , 14 : 1 (1969) pp. 170–173 Teoriya Veroyatnost. i ee Primenen. , 14 : 1 (1969) pp. 175–178