Namespaces
Variants
Actions

Difference between revisions of "Linear estimator"

From Encyclopedia of Mathematics
Jump to: navigation, search
m (tex encoded by computer)
(details)
 
Line 22: Line 22:
 
$$  
 
$$  
 
\widehat{ {\pmb\theta }}  = \  
 
\widehat{ {\pmb\theta }}  = \  
( \mathbf X  ^  \prime  \mathbf X ) ^ {-} 1
+
( \mathbf X  ^  \prime  \mathbf X )^{-1}
 
\mathbf X  ^  \prime  \mathbf Y
 
\mathbf X  ^  \prime  \mathbf Y
 
$$
 
$$
Line 34: Line 34:
 
is the  $  n \times p $
 
is the  $  n \times p $
 
matrix (of rank  $  p $)  
 
matrix (of rank  $  p $)  
of observed values  $  x _ {i} ^ {(} k) $,  
+
of observed values  $  x _ {i}^{(k)} $,  
 
$  i = 1 \dots n $,  
 
$  i = 1 \dots n $,  
 
$  k = 1 \dots p $,  
 
$  k = 1 \dots p $,  
Line 46: Line 46:
 
is the  $  n $-
 
is the  $  n $-
 
dimensional random column vector of residual components, which satisfies the condition  $  {\mathsf E} \epsilon = 0 $,  
 
dimensional random column vector of residual components, which satisfies the condition  $  {\mathsf E} \epsilon = 0 $,  
$  {\mathsf E} ( \epsilon \epsilon ^ \prime  ) = \sigma ^ {2} I $(
+
$  {\mathsf E} ( \epsilon \epsilon^\prime  ) = \sigma^{2} I $(
 
$  I $
 
$  I $
 
being the unit matrix).
 
being the unit matrix).
  
 
====References====
 
====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top"> H. Cramér,   "Mathematical methods of statistics" , Princeton Univ. Press  (1946)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top"> C.R. Rao,   "Linear statistical inference and its applications" , Wiley  (1965)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top"> S. Zacks,   "The theory of statistical inference" , Wiley  (1971)</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top"> H. Scheffé,   "Analysis of variance" , Wiley  (1959)</TD></TR></table>
+
<table>
 +
<TR><TD valign="top">[1]</TD> <TD valign="top"> H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press  (1946)</TD></TR>
 +
<TR><TD valign="top">[2]</TD> <TD valign="top"> C.R. Rao, "Linear statistical inference and its applications" , Wiley  (1965)</TD></TR>
 +
<TR><TD valign="top">[3]</TD> <TD valign="top"> S. Zacks, "The theory of statistical inference" , Wiley  (1971)</TD></TR>
 +
<TR><TD valign="top">[4]</TD> <TD valign="top"> H. Scheffé, "Analysis of variance" , Wiley  (1959)</TD></TR>
 +
</table>
  
 
====Comments====
 
====Comments====
 
When  $  \epsilon $
 
When  $  \epsilon $
is multivariate normally distributed, the least-squares estimator is actually optimal in a wider class of estimators: it is the minimum-variance [[Unbiased estimator|unbiased estimator]]. Even without the normality assumption it is asymptotically efficient ( $  n \rightarrow \infty $)  
+
is multivariate normally distributed, the least-squares estimator is actually optimal in a wider class of estimators: it is the minimum-variance [[unbiased estimator]]. Even without the normality assumption it is asymptotically efficient ( $  n \rightarrow \infty $)  
(cf. [[Efficiency, asymptotic|Efficiency, asymptotic]]) among all estimators.
+
(cf. [[Efficiency, asymptotic]]) among all estimators.

Latest revision as of 15:40, 7 July 2024


A linear function of observable random variables, used (when the actual values of the observed variables are substituted into it) as an approximate value (estimate) of an unknown parameter of the stochastic model under analysis (see Statistical estimator). The special selection of the class of linear estimators is justified for the following reasons. Linear estimators lend themselves more easily to statistical analysis, in particular to the investigation of consistency, unbiasedness, efficiency, the construction of corresponding confidence intervals, etc. At the same time, in a fairly wide range of cases the search for "optimal" (in a well-defined sense) estimators does not lead beyond the limits of the class of linear estimators. For example, the statistical analysis of a linear regression model (see Linear regression) of the form

$$ \mathbf Y = \mathbf X \pmb\theta + \epsilon $$

gives as best linear unbiased estimator of the parameter $ \pmb\theta $ the least-squares estimator

$$ \widehat{ {\pmb\theta }} = \ ( \mathbf X ^ \prime \mathbf X )^{-1} \mathbf X ^ \prime \mathbf Y $$

(linear with respect to the observed values of the random variable $ \mathbf Y $ under investigation). Here $ \mathbf Y $ is the $ n $- dimensional column vector of observed values $ y _ {i} $, $ i = 1 \dots n $, of the resulting test (random variable) under investigation, $ \mathbf X $ is the $ n \times p $ matrix (of rank $ p $) of observed values $ x _ {i}^{(k)} $, $ i = 1 \dots n $, $ k = 1 \dots p $, of $ p $ non-random factor arguments on which the resulting test $ \mathbf Y $ depends, $ \pmb\theta $ is the $ p $- dimensional column vector of the unknown parameters $ Q _ {k} $, $ k = 1 \dots p $, and $ \epsilon $ is the $ n $- dimensional random column vector of residual components, which satisfies the condition $ {\mathsf E} \epsilon = 0 $, $ {\mathsf E} ( \epsilon \epsilon^\prime ) = \sigma^{2} I $( $ I $ being the unit matrix).

References

[1] H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
[2] C.R. Rao, "Linear statistical inference and its applications" , Wiley (1965)
[3] S. Zacks, "The theory of statistical inference" , Wiley (1971)
[4] H. Scheffé, "Analysis of variance" , Wiley (1959)

Comments

When $ \epsilon $ is multivariate normally distributed, the least-squares estimator is actually optimal in a wider class of estimators: it is the minimum-variance unbiased estimator. Even without the normality assumption it is asymptotically efficient ( $ n \rightarrow \infty $) (cf. Efficiency, asymptotic) among all estimators.

How to Cite This Entry:
Linear estimator. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Linear_estimator&oldid=47655
This article was adapted from an original article by S.A. Aivazyan (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article