Efficiency, asymptotic

of a test

A concept that makes it possible in the case of large samples to make a quantitative comparison of two distinct statistical tests for a certain statistical hypothesis. The need to measure the efficiency of tests arose in the 1930s and -forties when simple (from the computational point of view) but "inefficient" rank procedures made their appearance.

There are several distinct approaches to the definition of the asymptotic efficiency of a test. Suppose that a distribution of observations is defined by a real parameter and that it is required to verify the hypothesis : against the alternative : . Suppose also that for a certain test with significance level there are observations needed to achieve a power against the given alternative and that another test of the same level needs for this purpose observations. Then one can define the relative efficiency of the first test with respect to the second by the formula . The concept of relative efficiency gives exhaustive information for the comparison of tests, but proves to be inconvenient for applications, since is a function of the three arguments , and and, as a rule, does not lend itself to computation in explicit form. To overcome this difficulty one uses a passage to a limit.

The quantity , for fixed and (if the limit exists), is called the asymptotic relative efficiency in the sense of Pitman. Similarly one defines the asymptotic relative efficiency in the sense of Bahadur, where for fixed , the limit is taken as tends to zero, and the asymptotic relative efficiency in the sense of Hodges and Lehmann, when for fixed and one computes the limit as .

Each of these definitions has its own merits and shortfalls. For example, the Pitman efficiency is, as a rule, easier to calculate than the Bahadur one (the calculation of the latter involves the non-trivial problem of studying the asymptotic probability of large deviations of test statistics); however, in a number of cases it turns out to be a less sensitive tool for the comparison of two tests.

Suppose, for example, that the observations are distributed according to the normal law with average and variance 1 and that the hypothesis : is to be verified against the alternative : . Suppose also that one considers a significance test based on a sample mean and Student ratio . Since the -test does not use information on the variance, the optimal test must be that based on . However, from the point of view of Pitman efficiency these tests are equivalent. On the other hand, the Bahadur efficiency of the -test in relation to is strictly less than 1 for any .

In more complicated cases the Pitman efficiency may depend on or and its calculation becomes very tedious. Then one calculates its limiting value as or . The latter usually is the same as the limiting value of the Bahadur efficiency as [8].

For other approaches to the definition of asymptotic efficiency of a test see [2]–[5]; sequential analogues of this concept are introduced in [6]–[7]. The choice of one definition or another must be based on which of them gives a more accurate approximation to the relative efficiency ; however, at present (1988) little is known in this direction [9].

References

[1]	A. Stewart, "The advanced theory of statistics" , 2. Inference and relationship , Griffin (1973)
[2]	R. Bahadur, "Rates of convergence of estimates and test statistics" Ann. Math. Stat. , 38 : 2 (1967) pp. 303–324
[3]	J. Hodges, E. Lehmann, "The efficiency of some nonparametric competitors of the -test" Ann. Math. Stat. , 27 : 2 (1956) pp. 324–335
[4]	C.R. Rao, "Linear statistical inference and its applications" , Wiley (1965)
[5]	W. Kallenberg, "Chernoff efficiency and deficiency" Ann. Statist. , 10 : 2 (1982) pp. 583–594
[6]	R. Berk, L. Brown, "Sequential Bahadur efficiency" Ann. Statist. , 6 : 3 (1978) pp. 567–581
[7]	R. Berk, "Asymptotic efficiencies of sequential tests" Ann. Statist. , 4 : 5 (1976) pp. 891–911
[8]	H. Wieland, "A condition under which the Pitman and Bahadur approaches to efficiency coincide" Ann. Statist. , 4 : 5 (1976) pp. 1003–1011
[9]	P. Groeneboom, J. Oosterhoff, "Bahadur efficiency and small-sample efficiency" Internat. Stat. Rev. , 49 : 2 (1981) pp. 127–141

Comments

Reference [a1] (and other work) suggest that, in the practically important case of small sample situations, the Pitman approach yields, in general, better approximations than the Bahadur approach does.

References

[a1]	P. Groeneboom, J. Oosterhoff, "Bahadur efficiencies and probabilities of large deviations" Stat. Neerlandica , 31 (1977) pp. 1–24