Efficiency, asymptotic

of a test

A concept that makes it possible in the case of large samples to make a quantitative comparison of two distinct statistical tests for a certain statistical hypothesis. The need to measure the efficiency of tests arose in the 1930s and -forties when simple (from the computational point of view) but "inefficient" rank procedures made their appearance.

There are several distinct approaches to the definition of the asymptotic efficiency of a test. Suppose that a distribution of observations is defined by a real parameter $ \theta $ and that it is required to verify the hypothesis $ H _ {0} $: $ \theta = \theta _ {0} $ against the alternative $ H _ {1} $: $ \theta \neq \theta _ {0} $. Suppose also that for a certain test with significance level $ \alpha $ there are $ N _ {1} $ observations needed to achieve a power $ \beta $ against the given alternative $ \theta $ and that another test of the same level needs for this purpose $ N _ {2} $ observations. Then one can define the relative efficiency of the first test with respect to the second by the formula $ e _ {12} = N _ {2} / N _ {1} $. The concept of relative efficiency gives exhaustive information for the comparison of tests, but proves to be inconvenient for applications, since $ e _ {12} $ is a function of the three arguments $ \alpha $, $ \beta $ and $ \theta $ and, as a rule, does not lend itself to computation in explicit form. To overcome this difficulty one uses a passage to a limit.

The quantity $ \lim\limits _ {\theta \rightarrow \theta _ {0} } e _ {12} ( \alpha , \beta , \theta ) $, for fixed $ \alpha $ and $ \beta $( if the limit exists), is called the asymptotic relative efficiency in the sense of Pitman. Similarly one defines the asymptotic relative efficiency in the sense of Bahadur, where for fixed $ \beta $, $ \theta $ the limit is taken as $ \alpha $ tends to zero, and the asymptotic relative efficiency in the sense of Hodges and Lehmann, when for fixed $ \alpha $ and $ \theta $ one computes the limit as $ \beta \rightarrow 1 $.

Each of these definitions has its own merits and shortfalls. For example, the Pitman efficiency is, as a rule, easier to calculate than the Bahadur one (the calculation of the latter involves the non-trivial problem of studying the asymptotic probability of large deviations of test statistics); however, in a number of cases it turns out to be a less sensitive tool for the comparison of two tests.

Suppose, for example, that the observations are distributed according to the normal law with average $ \theta $ and variance 1 and that the hypothesis $ H _ {0} $: $ \theta = 0 $ is to be verified against the alternative $ H _ {1} $: $ \theta > 0 $. Suppose also that one considers a significance test based on a sample mean $ \overline{X}\; $ and Student ratio $ t $. Since the $ t $- test does not use information on the variance, the optimal test must be that based on $ \overline{X}\; $. However, from the point of view of Pitman efficiency these tests are equivalent. On the other hand, the Bahadur efficiency of the $ t $- test in relation to $ \overline{X}\; $ is strictly less than 1 for any $ \theta > 0 $.

In more complicated cases the Pitman efficiency may depend on $ \alpha $ or $ \beta $ and its calculation becomes very tedious. Then one calculates its limiting value as $ \beta \rightarrow 1 $ or $ \alpha \rightarrow 0 $. The latter usually is the same as the limiting value of the Bahadur efficiency as $ \theta \rightarrow \theta _ {0} $[8].

For other approaches to the definition of asymptotic efficiency of a test see [2]–[5]; sequential analogues of this concept are introduced in [6]–[7]. The choice of one definition or another must be based on which of them gives a more accurate approximation to the relative efficiency $ e _ {12} $; however, at present (1988) little is known in this direction [9].

References

[1]	A. Stewart, "The advanced theory of statistics" , 2. Inference and relationship , Griffin (1973)
[2]	R. Bahadur, "Rates of convergence of estimates and test statistics" Ann. Math. Stat. , 38 : 2 (1967) pp. 303–324
[3]	J. Hodges, E. Lehmann, "The efficiency of some nonparametric competitors of the -test" Ann. Math. Stat. , 27 : 2 (1956) pp. 324–335
[4]	C.R. Rao, "Linear statistical inference and its applications" , Wiley (1965)
[5]	W. Kallenberg, "Chernoff efficiency and deficiency" Ann. Statist. , 10 : 2 (1982) pp. 583–594
[6]	R. Berk, L. Brown, "Sequential Bahadur efficiency" Ann. Statist. , 6 : 3 (1978) pp. 567–581
[7]	R. Berk, "Asymptotic efficiencies of sequential tests" Ann. Statist. , 4 : 5 (1976) pp. 891–911
[8]	H. Wieland, "A condition under which the Pitman and Bahadur approaches to efficiency coincide" Ann. Statist. , 4 : 5 (1976) pp. 1003–1011
[9]	P. Groeneboom, J. Oosterhoff, "Bahadur efficiency and small-sample efficiency" Internat. Stat. Rev. , 49 : 2 (1981) pp. 127–141

Comments

Reference [a1] (and other work) suggest that, in the practically important case of small sample situations, the Pitman approach yields, in general, better approximations than the Bahadur approach does.

References

[a1]	P. Groeneboom, J. Oosterhoff, "Bahadur efficiencies and probabilities of large deviations" Stat. Neerlandica , 31 (1977) pp. 1–24