Consistent test

consistent statistical test

A statistical test that reliably distinguishes a hypothesis to be tested from an alternative by increasing the number of observations to infinity.

Let be a sequence of independent identically-distributed random variables taking values in a sample space , , and suppose one is testing the hypothesis : against the alternative : , with an error of the first kind (see Significance level) being given in advance and equal to (). Suppose that the first observations are used to construct a statistical test of level for testing against , and let , , be its power function (cf. Power function of a test), which gives for every the probability that this test rejects when the random variable is subject to the law . Of course for all . By increasing the number of observations without limit it is possible to construct a sequence of statistical tests of a prescribed level intended to test against ; the corresponding sequence of power functions satisfies the condition

If under these conditions the sequence of power functions is such that, for any fixed ,

then one says that a consistent sequence of statistical tests of level has been constructed for testing against . With a certain amount of license, one says that a consistent test has been constructed. Since , (which is the restriction of , , to ), is the power of the statistical test constructed from the observations , the property of consistency of a sequence of statistical tests can be expressed as follows: The corresponding powers , , converge on to the function identically equal to 1 on .

Example. Let be independent identically-distributed random variables whose distribution function belongs to the family of all continuous distribution functions on , and let be a vector of positive probabilities such that . Further, let be any distribution function of . Then and uniquely determine a partition of the real axis into intervals , where

In other words, the end points of the intervals are quantiles of the distribution function . These intervals determine a partition of into two disjoint sets and as follows: A distribution function of belongs to if and only if

and otherwise . Now let be the vector of counts obtained as a result of grouping the first random variables () into the intervals . Then to test the hypothesis that the distribution function of the belongs to the set against the alternative that it belongs to the set , one can make use of the "chi-squared" test based on the statistic

According to this, with significance level (), the hypothesis must be rejected whenever , where is the upper -quantile of the "chi-squared" distribution with degrees of freedom. From the general theory of tests of "chi-squared" type it follows that when is correct,

which also shows the consistency of the "chi-squared" test for testing against . But if one takes an arbitrary non-empty subset of and considers the problem of testing against the alternative , then it is clear that the "chi-squared" sequence of tests based on the statistics is not consistent, since

and, in particular,

References

[1]	S.S. Wilks, "Mathematical statistics" , Wiley (1962)
[2]	E. Lehman, "Testing statistical hypotheses" , Wiley (1959)