View source for Linear estimator

<!--
l0592001.png
$#A+1 = 27 n = 0
$#C+1 = 27 : ~/encyclopedia/old_files/data/L059/L.0509200 Linear estimator
Automatically converted into TeX, above some diagnostics.
Please remove this comment and the {{TEX|auto}} line below,
if TeX found to be correct.
-->

{{TEX|auto}}
{{TEX|done}}

A linear function of observable random variables, used (when the actual values of the observed variables are substituted into it) as an approximate value (estimate) of an unknown parameter of the stochastic model under analysis (see [[Statistical estimator|Statistical estimator]]). The special selection of the class of linear estimators is justified for the following reasons. Linear estimators lend themselves more easily to statistical analysis, in particular to the investigation of consistency, unbiasedness, efficiency, the construction of corresponding confidence intervals, etc. At the same time, in a fairly wide range of cases the search for  "optimal"  (in a well-defined sense) estimators does not lead beyond the limits of the class of linear estimators. For example, the statistical analysis of a linear regression model (see [[Linear regression|Linear regression]]) of the form

$$ 
\mathbf Y  =  \mathbf X \pmb\theta + \epsilon
$$

gives as best linear unbiased estimator of the parameter  $  \pmb\theta $
the least-squares estimator

$$ 
\widehat{ {\pmb\theta }}   = \ 
( \mathbf X  ^  \prime  \mathbf X )  ^ {-} 1
\mathbf X  ^  \prime  \mathbf Y
$$

(linear with respect to the observed values of the random variable  $  \mathbf Y $
under investigation). Here  $  \mathbf Y $
is the  $  n $-
dimensional column vector of observed values  $  y _ {i} $, 
$  i = 1 \dots n $, 
of the resulting test (random variable) under investigation,  $  \mathbf X $
is the  $  n \times p $
matrix (of rank  $  p $) 
of observed values  $  x _ {i}  ^ {(} k) $, 
$  i = 1 \dots n $, 
$  k = 1 \dots p $, 
of  $  p $
non-random factor arguments on which the resulting test  $  \mathbf Y $
depends,  $  \pmb\theta $
is the  $  p $-
dimensional column vector of the unknown parameters  $  Q _ {k} $, 
$  k = 1 \dots p $, 
and  $  \epsilon $
is the  $  n $-
dimensional random column vector of residual components, which satisfies the condition  $  {\mathsf E} \epsilon = 0 $, 
$  {\mathsf E} ( \epsilon \epsilon  ^  \prime  ) = \sigma  ^ {2} I $(
$  I $
being the unit matrix).

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  H. Cramér,   "Mathematical methods of statistics" , Princeton Univ. Press  (1946)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  C.R. Rao,   "Linear statistical inference and its applications" , Wiley  (1965)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  S. Zacks,   "The theory of statistical inference" , Wiley  (1971)</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  H. Scheffé,   "Analysis of variance" , Wiley  (1959)</TD></TR></table>

====Comments====
When  $  \epsilon $
is multivariate normally distributed, the least-squares estimator is actually optimal in a wider class of estimators: it is the minimum-variance [[Unbiased estimator|unbiased estimator]]. Even without the normality assumption it is asymptotically efficient ( $  n \rightarrow \infty $) 
(cf. [[Efficiency, asymptotic|Efficiency, asymptotic]]) among all estimators.