# Efficiency, asymptotic

of a test

A concept that makes it possible in the case of large samples to make a quantitative comparison of two distinct statistical tests for a certain statistical hypothesis. The need to measure the efficiency of tests arose in the 1930s and -forties when simple (from the computational point of view) but "inefficient" rank procedures made their appearance.

There are several distinct approaches to the definition of the asymptotic efficiency of a test. Suppose that a distribution of observations is defined by a real parameter $\theta$ and that it is required to verify the hypothesis $H _ {0}$: $\theta = \theta _ {0}$ against the alternative $H _ {1}$: $\theta \neq \theta _ {0}$. Suppose also that for a certain test with significance level $\alpha$ there are $N _ {1}$ observations needed to achieve a power $\beta$ against the given alternative $\theta$ and that another test of the same level needs for this purpose $N _ {2}$ observations. Then one can define the relative efficiency of the first test with respect to the second by the formula $e _ {12} = N _ {2} / N _ {1}$. The concept of relative efficiency gives exhaustive information for the comparison of tests, but proves to be inconvenient for applications, since $e _ {12}$ is a function of the three arguments $\alpha$, $\beta$ and $\theta$ and, as a rule, does not lend itself to computation in explicit form. To overcome this difficulty one uses a passage to a limit.

The quantity $\lim\limits _ {\theta \rightarrow \theta _ {0} } e _ {12} ( \alpha , \beta , \theta )$, for fixed $\alpha$ and $\beta$( if the limit exists), is called the asymptotic relative efficiency in the sense of Pitman. Similarly one defines the asymptotic relative efficiency in the sense of Bahadur, where for fixed $\beta$, $\theta$ the limit is taken as $\alpha$ tends to zero, and the asymptotic relative efficiency in the sense of Hodges and Lehmann, when for fixed $\alpha$ and $\theta$ one computes the limit as $\beta \rightarrow 1$.

Each of these definitions has its own merits and shortfalls. For example, the Pitman efficiency is, as a rule, easier to calculate than the Bahadur one (the calculation of the latter involves the non-trivial problem of studying the asymptotic probability of large deviations of test statistics); however, in a number of cases it turns out to be a less sensitive tool for the comparison of two tests.

Suppose, for example, that the observations are distributed according to the normal law with average $\theta$ and variance 1 and that the hypothesis $H _ {0}$: $\theta = 0$ is to be verified against the alternative $H _ {1}$: $\theta > 0$. Suppose also that one considers a significance test based on a sample mean $\overline{X}\;$ and Student ratio $t$. Since the $t$- test does not use information on the variance, the optimal test must be that based on $\overline{X}\;$. However, from the point of view of Pitman efficiency these tests are equivalent. On the other hand, the Bahadur efficiency of the $t$- test in relation to $\overline{X}\;$ is strictly less than 1 for any $\theta > 0$.

In more complicated cases the Pitman efficiency may depend on $\alpha$ or $\beta$ and its calculation becomes very tedious. Then one calculates its limiting value as $\beta \rightarrow 1$ or $\alpha \rightarrow 0$. The latter usually is the same as the limiting value of the Bahadur efficiency as $\theta \rightarrow \theta _ {0}$[8].

For other approaches to the definition of asymptotic efficiency of a test see [2][5]; sequential analogues of this concept are introduced in [6][7]. The choice of one definition or another must be based on which of them gives a more accurate approximation to the relative efficiency $e _ {12}$; however, at present (1988) little is known in this direction [9].

#### References

 [1] A. Stewart, "The advanced theory of statistics" , 2. Inference and relationship , Griffin (1973) [2] R. Bahadur, "Rates of convergence of estimates and test statistics" Ann. Math. Stat. , 38 : 2 (1967) pp. 303–324 [3] J. Hodges, E. Lehmann, "The efficiency of some nonparametric competitors of the -test" Ann. Math. Stat. , 27 : 2 (1956) pp. 324–335 [4] C.R. Rao, "Linear statistical inference and its applications" , Wiley (1965) [5] W. Kallenberg, "Chernoff efficiency and deficiency" Ann. Statist. , 10 : 2 (1982) pp. 583–594 [6] R. Berk, L. Brown, "Sequential Bahadur efficiency" Ann. Statist. , 6 : 3 (1978) pp. 567–581 [7] R. Berk, "Asymptotic efficiencies of sequential tests" Ann. Statist. , 4 : 5 (1976) pp. 891–911 [8] H. Wieland, "A condition under which the Pitman and Bahadur approaches to efficiency coincide" Ann. Statist. , 4 : 5 (1976) pp. 1003–1011 [9] P. Groeneboom, J. Oosterhoff, "Bahadur efficiency and small-sample efficiency" Internat. Stat. Rev. , 49 : 2 (1981) pp. 127–141