View source for Asymptotic optimality

''of estimating functions''

Efficient estimation (cf. [[Efficient estimator|Efficient estimator]]) of parameters in stochastic models is most conveniently approached via properties of estimating functions, namely functions of the data and the parameter of interest, rather than estimators derived therefrom. For a detailed explanation see [[#References|[a1]]], Chapt. 1.

Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107901.png" /> be a [[Sample|sample]] in discrete or continuous time from a stochastic system taking values in an <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107902.png" />-dimensional Euclidean space. The distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107903.png" /> depends on a parameter of interest <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107904.png" /> taking values in an open subset of a <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107905.png" />-dimensional Euclidean space. The possible probability measures (cf. [[Probability measure|Probability measure]]) for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107906.png" /> are <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107907.png" />, a union of families of models.

Consider the class <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107908.png" /> of zero-mean square-integrable estimating functions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a1107909.png" />, which are vectors of dimension <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079010.png" /> and for which the matrices used below are non-singular.

Optimality in both the fixed sample and the asymptotic sense is considered. The former involves choice of an estimating function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079011.png" /> to maximize, in the partial order of non-negative definite matrices, the information criterion

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079012.png" /></td> </tr></table>

which is a natural generalization of the [[Fisher amount of information|Fisher amount of information]]. Here <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079013.png" /> is the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079014.png" />-matrix of derivatives of the elements of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079015.png" /> with respect to those of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079016.png" /> and prime denotes transposition. If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079017.png" /> is a prespecified family of estimating functions, it is said that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079018.png" /> is fixed sample optimal in <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079019.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079020.png" /> is non-negative definite for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079021.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079022.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079023.png" />. Then, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079024.png" /> is the element of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079025.png" /> whose dispersion distance from the maximum information estimating function in <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079026.png" /> (often the likelihood score) is least.

A focus on asymptotic properties can be made by confining attention to the subset <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079027.png" /> of estimating functions which are martingales (cf. [[Martingale|Martingale]]). Here one considers <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079028.png" /> ranging over the positive real numbers and for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079029.png" /> one writes <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079030.png" /> for the quadratic characteristic, the predictable increasing process for which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079031.png" /> is a martingale. Also, write <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079032.png" /> for the predictable process for which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079033.png" /> is a martingale. Then, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079034.png" /> is asymptotically optimal in <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079035.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079036.png" /> is almost surely non-negative definite for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079037.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079038.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079039.png" />, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079040.png" />, where

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079041.png" /></td> </tr></table>

Under suitable regularity conditions, asymptotically optimal estimating functions produce estimators for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/a/a110/a110790/a11079042.png" /> which are consistent (cf. [[Consistent estimator|Consistent estimator]]), asymptotically unbiased (cf. [[Unbiased estimator|Unbiased estimator]]) and asymptotically normally distributed (cf. [[Normal distribution|Normal distribution]]) with minimum size asymptotic confidence zones (cf. [[Confidence estimation|Confidence estimation]]). For further details see [[#References|[a2]]], [[#References|[a3]]].

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  D.L. McLeish,   C.G. Small,   "The theory and applications of statistical inference functions" , ''Lecture Notes in Statistics'' , Springer  (1988)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  V.P. Godambe,   C.C. Heyde,   "Quasi-likelihood and optimal estimation"  ''Internat. Statist. Rev.'' , '''55'''  (1987)  pp. 231–244.</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  C.C. Heyde,   "Quasi-likelihood and its application. A general approach to optimal parameter estimation" , Springer  (1997)</TD></TR></table>