Significance test

One of the basic methods for statistical hypotheses testing (cf. Statistical hypotheses, verification of), used to confront the outcomes of observations , interpreted as realizations of random variables , with a hypothesis . It is usually based on the observed value of a suitable statistic , the specific form of which depends on the formulation of the problem. Generally speaking, the application of a significance test does not presuppose the availability of an alternative hypothesis , to be accepted if is rejected; however, if some is specified, then, according to the general theory of statistical hypotheses testing, it is precisely that determines the choice of the statistic , the governing principle being to maximize the power of the test (cf. Power of a statistical test).

Significance tests are usually applied as follows. Having selected the test by choosing in advance a statistic and a significance level , , determine the critical value of the test by the condition . According to the significance test with level , the hypothesis is rejected if . If , the conclusion is that is not in disagreement with the observations , at least as long as new observations do not oblige the experimenter to adopt a different point of view.

Example. A counter monitoring a Poisson process registers 150 pulses during the first hour, 117 pulses during the second. Can it be assumed on this basis that the pulse arrival rate per unit time is constant (hypothesis )?

If is true, the observed values 150, 117 may be treated as realizations of two independent random variables and obeying the same Poisson law with parameter ; the value of is unknown. Since implies that the random variables

are approximately normally distributed with parameters , it follows that the statistic

is approximately distributed according to a "chi-squared" law with one degree of freedom, i.e.

Tables of the "chi-squared" distribution yield the critical value for the prescribed significance level , i.e.

Now, using the observed values and , one computes the value of the test statistic:

Since , it follows that the hypothesis (constant pulse arrival rate) is rejected by the "chi-squared" test at significance level .

References

[1]	H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
[2]	E.L. Lehmann, "Testing statistical hypotheses" , Wiley (1959)
[3]	N.V. Smirnov, I.V. Dunin-Barkovskii, "Mathematische Statistik in der Technik" , Deutsch. Verlag Wissenschaft. (1969) (Translated from Russian)
[4]	B.I. Devyatov, "Limits of admissibility of normal approximations to the Poisson distribution" Theor. Probab. Appl. , 14 : 1 (1969) pp. 170–173 Teoriya Veroyatnost. i ee Primenen. , 14 : 1 (1969) pp. 175–178