Intermediate efficiency

Kallenberg efficiency

A concept used to compare the performance of statistical tests (cf. also Statistical hypotheses, verification of). Write for the sample size required to attain with a level- test a prescribed power at an alternative . If one has two tests with corresponding numbers and , respectively, the ratio is called the relative efficiency of test with respect to test . If the relative efficiency equals , test needs times as much observations to perform equally well as test and hence test is times as efficient as test (cf. also Efficient test).

In general, the relative efficiency is hard to compute and, if it can be computed, hard to evaluate, as it depends on three arguments: , and . (Note that is not restricted to be a Euclidean parameter; it can also be an abstract parameter, as for instance the distribution function.) Therefore, an asymptotic approach, where tends to infinity, is welcome to simplify both the computation and interpretation, thus hoping that the limit gives a sufficiently good approximation of the far more complicated finite-sample case.

When sending to infinity, two guiding principles are:

a) to "decrease the significance probability as N increases" , i.e. to send to ; or

b) to "move the alternative hypothesis steadily closer to the null hypothesis" , i.e. to send to . Both principles are attractive: with more observations it seems reasonable to have a stronger requirement on the level and, on the other hand, for alternatives far away from the null hypothesis there is no need for statistical methods, since they are obviously different from .

In Pitman's asymptotic efficiency concept, method b) is used, while one deals with fixed levels, thus ignoring principle a). In Bahadur's asymptotic efficiency concept, method a) is actually used, while one considers fixed alternatives, thereby ignoring principle b). (Cf. also Bahadur efficiency; Efficiency, asymptotic.) Intermediate or Kallenberg efficiency applies both attractive principles simultaneously.

As a consequence of Bahadur's approach, in typical cases the level of significance required to attain a fixed power at a fixed alternative tends to zero at an exponential rate as the number of observations tends to infinity. There remains a whole range of sequences of levels "intermediate" between these two extremes of very fast convergence to zero of and the fixed in the case of Pitman efficiency. The efficiency concept introduced by W.C.M. Kallenberg [a11] deals with this intermediate range and is therefore called intermediate efficiency, or, for short, i-efficiency.

A related approach is applied by P. Groeneboom [a5], Sect. 3.4, studying very precisely the behaviour of several tests for the multivariate linear hypothesis from an "intermediate" point of view. Other efficiency concepts with an "intermediate" flavour can be found in [a6], [a18] and [a2].

Instead of applying principles a) and b) simultaneously, in a lot of papers they are applied one after the other. For an excellent treatment in the case of non-parametric tests see [a16], where also many further references can be found (cf. also Non-parametric test). General results on limiting equivalence of local and non-local measures of efficiency are presented in [a4], [a14] and [a13].

The definition of intermediate or Kallenberg efficiency is as follows. Let be a sequence of independent, identically distributed random variables with distribution for some in the parameter space . The hypothesis : has to be tested against : , where and are given subsets of . For a family of tests , denote the power at by , where is the available number of observations and is the level of the test (cf. also Significance level). Suppose one has two families of tests, and . Let be a sequence of levels with

(a1)

thus ensuring that tends to , but not exponentially fast. Let be a sequence of alternatives tending to the null hypothesis, in the sense that

(a2)

and

(a3)

Here, and denotes the Hellinger distance between the probability measures and . This ensures that the alternatives tend to , but in a slower way than contiguous alternatives, cf. [a17]. Typically, for Euclidean parameters, as and hence in such cases formula (a3) concerns convergence of to at a rate slower than . (The latter is the usual rate for contiguous alternatives.)

Define as the smallest number of observations needed for to perform as well as in the sense that , the power at of the level- test of based on observations, is, for all , at least as large as , the power at of the level- test of based on observations. If the sequence of levels satisfies next to (a1) also

(a4)

and if

exists and does not depend on the special sequences , under consideration, one says that the intermediate or Kallenberg efficiency of with respect to equals . If (a4) is replaced by

one speaks of weak intermediate or weak Kallenberg efficiency of with respect to and one uses the notation . Otherwise, that is, if all sequences satisfying (a1) are under consideration, one speaks of strong intermediate or strong Kallenberg efficiency of with respect to , with notation . Note that

So, the whole intermediate range of levels between the Pitman and Bahadur case is built up with three increasing ranges. For example, if an i-efficiency result can be proved only for at a lower rate than powers of , that is, , one speaks of a weak i-efficiency result. The several types of i-efficiency correspond with the existence of several types of moderate and Cramér-type large deviation theorems.

To compute under the null hypothesis, one needs a moderate deviation result (see [a7] and references therein for results of this type), since tends to . Under the alternatives a kind of law of large numbers is involved. The precise computation is described in [a11], Lemma 2.1; Corol. 2.2, where also many examples are presented.

In many testing problems, likelihood-ratio tests (cf. also Likelihood-ratio test) are asymptotically optimal (cf. also Asymptotic optimality) when comparison is made in a non-local way, cf. [a1], [a3], [a10]. On the other hand, likelihood ratio tests usually are not asymptotically optimal with respect to criteria based on the local performance of tests. It turns out that in exponential families, likelihood ratio tests have strong i-efficiency greater than or equal to one with respect to every other test, thus being optimal according to the criterion of i-efficiency.

Locally most powerful tests are often Pitman efficient. On the other hand, locally most powerful tests are far from optimal from a non-local point of view. It turns out that in curved exponential families locally most powerful tests have strong i-efficiency greater than or equal to one with respect to every other test, thus being optimal according to the criterion of i-efficiency.

Optimality, in the sense of weak i-efficiency, of certain goodness-of-fit tests (cf. also Goodness-of-fit test) in the case of censored data is shown in [a15], while i-efficiency of decomposable statistics in a multinomial scheme is analyzed in [a9]. For a generalization of the concept see [a8], where it is shown that data-driven Neyman tests are asymptotically optimal.

Application of an intermediate approach in estimation theory can be found in [a12]. This is based on the probability that a statistical estimator deviates by more than from its target , for instance for the estimator . The intermediate range concerns and . Under certain regularity conditions, there is an asymptotic lower bound for , similar to the (Fisher) information bound in the local theory. An estimator is called optimal in the intermediate sense if it attains this lower bound.

References

[a1]	R.R. Bahadur, "An optimal property of the likelihood ratio statistic" , Proc. 5th Berkeley Symp. Math. Stat. Probab. , 1 , Univ. California Press (1965) pp. 13–26
[a2]	A.A. Borovkov, A.A. Mogulskii, "Large deviations and statistical invariance principle" Th. Probab. Appl. , 37 (1993) pp. 7–13
[a3]	L.D. Brown, "Non-local asymptotic optimality of appropriate likelihood ratio tests" Ann. Math. Stat. , 42 (1971) pp. 1206–1240
[a4]	H.S. Wieand, "A condition under which the Pitman and Bahadur approaches to efficiency coincide" Ann. Statist. , 4 (1976) pp. 1003–1011
[a5]	P. Groeneboom, "Large deviations and asymptotic efficiencies" , Math. Centre Tracts , 118 , Math. Centre Amsterdam (1980)
[a6]	W. Hoeffding, "Asymptotic optimal tests for multinomial distributions" Ann. Math. Stat. , 36 (1965) pp. 369–405
[a7]	T. Inglot, W.C.M. Kallenberg, T. Ledwina, "Strong moderate deviation theorems" Ann. of Probab. , 20 (1992) pp. 987–1003
[a8]	T. Inglot, T. Ledwina, "Asymptotic optimality of data-driven Neyman's tests for uniformity" Ann. Statist. , 24 (1996) pp. 1982–2019
[a9]	G.I. Ivchenko, Sh.A. Mirakhemedov, "Large deviations and intermediate efficiency of decomposable statistics in a multinomial scheme" Math. Methods Statist. , 4 (1995) pp. 294–311
[a10]	W.C.M. Kallenberg, "Bahadur deficiency of likelihood ratio tests in exponential families" J. Multivariate Anal. , 11 (1981) pp. 506–531
[a11]	W.C.M. Kallenberg, "Intermediate efficiency, theory and examples" Ann. Statist. , 11 (1983) pp. 170–182
[a12]	W.C.M. Kallenberg, "On moderate deviation theory in estimation" Ann. Statist. , 11 (1983) pp. 498–504
[a13]	W.C.M. Kallenberg, A.J. Koning, "On Wieand's theorem" Statist. Probab. Lett. , 25 (1995) pp. 121–132
[a14]	W.C.M. Kallenberg, T. Ledwina, "On local and nonlocal measures of efficiency" Ann. Statist. , 15 (1987) pp. 1401–1420
[a15]	A.J. Koning, "Approximation of stochastic integrals with applications to goodness-of-fit tests" Ann. Statist. , 20 (1992) pp. 428–454
[a16]	Ya.Yu. Nikitin, "Asymptotic efficiency of nonparametric tests" , Cambridge Univ. Press (1995)
[a17]	J. Oosterhoff, W.R. van Zwet, "A note on contiguity and Hellinger distance" J. Jurečkova (ed.) , Contributions to Statistics: J. Hájek Memorial Vol. , Acad. Prague (1979) pp. 157–166
[a18]	H. Rubin, J. Sethuraman, "Bayes risk efficiency" Sankhyā Ser. A , 27 (1965) pp. 347–356