Efficiency of a statistical procedure

A concept used to compare statistical procedures in a given class with an optimal one. In mathematical statistics the notion of optimality of a statistical procedure is usually expressed in terms of the risk (function) of the procedure, which in turns depends directly on the choice of the loss function. Therefore, it can happen that one and the same statistical procedure is very efficient or even optimal in one sense and less efficient in another.

Efficiency of a statistical procedure is a concept not altogether clearly defined. It acquires a more precise meaning in specific problems of mathematical statistics such as, for example, the testing of statistical hypotheses and statistical estimation.

In broad terms, we can classify efficiency in 2 x 2 categories: Efficiency considerations are different for estimators and hypothesis tests, and also finite sample efficiency is different from asymptotic efficiency. For asymptotic efficiency of hypothesis tests, see entry on Efficiency, asymptotic. Finite sample efficiency of tests can be evaluated using the concept of stringency; Lehmann, Testing Statistical Hypothesis provides an exposition of this concept; the original text has been revised and updated – Lehman & Romano. For estimators, the basic tool is the Cramer-Rao Lower Bound (CRLB) on variances of unbiased estimators. In finite samples CRLB may not be sharp, and biased estimators may achieve greater efficiency. However, asymptotically, the CRLB provides a general bound on best possible performance of all regular estimators. Again, Lehman: Theory of Point Estimation provides a good exposition – this classic has also been revised and updated to Lehmann & Casella.