View source for Best linear unbiased estimator

''BLUE''

Let

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104401.png" /></td> <td valign="top" style="width:5%;text-align:right;">(a1)</td></tr></table>

be a [[Linear regression|linear regression]] model, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104402.png" /> is a random column vector of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104403.png" />  "measurements" , <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104404.png" /> is a known non-random  "plan"  matrix, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104405.png" /> is an unknown vector of the parameters, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104406.png" /> is a random  "error" , or  "noise" , vector with mean <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104407.png" /> and a possibly unknown non-singular covariance matrix <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104408.png" />. A model with linear restrictions on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b1104409.png" /> can be obviously reduced to (a1). Without loss of generality, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044010.png" />.

Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044011.png" />; a linear unbiased estimator (LUE) of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044012.png" /> is a [[Statistical estimator|statistical estimator]] of the form <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044013.png" /> for some non-random matrix <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044014.png" /> such that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044015.png" /> for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044016.png" />, i.e., <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044017.png" />. A linear [[Unbiased estimator|unbiased estimator]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044018.png" /> of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044019.png" /> is called a best linear unbiased estimator (BLUE) of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044020.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044021.png" /> for all linear unbiased estimators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044022.png" /> of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044023.png" />, i.e., if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044024.png" /> for all linear unbiased estimators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044025.png" /> of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044026.png" /> and all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044027.png" />.

Since it is assumed that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044028.png" />, there exists a unique best linear unbiased estimator of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044029.png" /> for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044030.png" />. It is then given by the formula <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044031.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044032.png" />, which coincides by the Gauss–Markov theorem (cf. [[Least squares, method of|Least squares, method of]]) with the least square estimator of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044033.png" />, defined as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044034.png" />; as usual, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044035.png" /> stands for transposition.

Because <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044036.png" /> is normally not known, Yu.A. Rozanov [[#References|[a2]]] has suggested to use a  "pseudo-best"  estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044037.png" /> in place of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044038.png" />, with an appropriately chosen <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044039.png" />. This idea has been further developed by A.M. Samarov [[#References|[a3]]] and I.F. Pinelis [[#References|[a4]]]. In particular, Pinelis has obtained duality theorems for the minimax risk and equations for the minimax solutions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044040.png" /> assumed to belong to an arbitrary known convex set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044041.png" /> of positive-definite <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044042.png" />-matrices with respect to the general quadratic risk function of the form

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044043.png" /></td> </tr></table>

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044044.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044045.png" /> is any non-negative-definite <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044046.png" />-matrix and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044047.png" /> stands for the expectation assuming <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110440/b11044048.png" />. Asymptotic versions of these results have also been given by Pinelis for the case when the  "noise"  is a second-order stationary stochastic process with an unknown spectral density belonging to an arbitrary, but known, convex class of spectral densities and by Samarov in the case of contamination classes.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  C.R. Rao,   "Linear statistical inference and its applications" , Wiley  (1965)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  Yu.A. Rozanov,   "On a new class of estimates" , ''Multivariate Analysis'' , '''2''' , Acad. Press  (1969)  pp. 437–441</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  A.M. Samarov,   "Robust spectral regression"  ''Ann. Math. Stat.'' , '''15'''  (1987)  pp. 99–111</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  I.F. Pinelis,   "On the minimax estimation of regression"  ''Th. Probab. Appl.'' , '''35'''  (1990)  pp. 500–512</TD></TR></table>