View source for Best linear unbiased estimation in linear models


{| class="wikitable"
!Copyright notice <!-- don't remove! --> 
|-
| This article ''Best linear unbiased estimation in linear models'' was adapted from an original article by Simo Puntanen, George P.H. Styan, which appeared in ''StatProb: The Encyclopedia Sponsored by Statistics and Probability Societies''. The original article ([<nowiki>http://statprob.com/encyclopedia/BestLinearUnbiasedEstimatinInLinearModels.html</nowiki> StatProb Source], Local Files: [[Media:best_linear_unbiased_estimation_in_linear_models.pdf|pdf]] | [[Media:best_linear_unbiased_estimation_in_linear_models.tex|tex]]) is copyrighted by the author(s), the article has been donated to ''Encyclopedia of Mathematics'', and its further issues are under ''Creative Commons Attribution Share-Alike License'. All pages from StatProb are contained in the [[:Category:Statprob|Category StatProb]].
|-
|}

$\def\mx#1{ {\mathbf{#1}}}$
$\def\BETA{\beta}\def\BETAH{ {\hat\beta}}\def\BETAT{ {\tilde\beta}}\def\betat{\tilde\beta}$
$\def\C{ {\mathscr C}}$
$\def\cov{\mathrm{cov}}\def\M{ {\mathscr M}}$
$\def\NS{ {\mathscr N}}\def\OLSE{ {\small\mathrm{OLSE}}}$
$\def\rank{ {\rm rank}} \def\tr{ { \rm trace}}$ 
$\def\rz{ {\mathbf{R}}} \def\SIGMA{\Sigma} \def\var{ {\rm var}}$
$\def\BLUE}{\small\mathrm{BLUE}}$
$\def\BLUP}{\small\mathrm{BLUP}}$
$\def\EPS{\varepsilon}$
$\def\EE{E}$
$\def\E{E}$
$\def\GAMMA{\gamma}$
{{MSC|62J05}}

<!-- \documentclass[10pt]{article} 
     \usepackage{amssymb} 
     \usepackage{amsmath} 
     \usepackage{amsfonts} 
     \usepackage{amsthm} 
     \usepackage{mathrsfs} 

$ \newcommand{\BETA}{ {\boldsymbol{\beta}}} \newcommand*{\BETAH}{{\boldsymbol{\hat\beta}}} \newcommand*{\BETAT}{{\boldsymbol{\tilde\beta}}} $
$ \newcommand{\BLUE}{\mbox{\small$\mathrm{BLUE}$}} $
$ \newcommand{\BLUP}{\mbox{\small$\mathrm{BLUP}$}} $
$ \newcommand{\betat}{\tilde\beta} $
$ \DeclareMathOperator{\C}{ {\mathscr C}} \DeclareMathOperator{\cov}{cov} \DeclareMathOperator{\EE}{E} \DeclareMathOperator{\E}{E} $ \newcommand{\EPS}{ {\boldsymbol{\varepsilon}}} \newcommand*{\GAMMA}{ {\boldsymbol{\gamma}}} \newcommand*{\M}{ {\mathscr M}} \newcommand*{\mx}[1]{\mathbf{#1}} \DeclareMathOperator{\NS}{ {\mathscr N}} \newcommand*{\OLSE}{\mbox{\small$\mathrm{OLSE}$}} $
$ \DeclareMathOperator{\rank}{rank} \DeclareMathOperator{\tr}{trace} $ \newcommand{\rz}{ {\mathbb{R}}} \newcommand*{\SIGMA}{ {\boldsymbol{\Sigma}}} \DeclareMathOperator{\var}{var} $

 \newtheorem{theorem}{Theorem}  
     \begin{document} -->
<center>'''Best Linear Unbiased Estimation in Linear Models'''</center>

<center> Simo Puntanen <ref> Department of Mathematics and Statistics,
FI-33014 University of Tampere, Tampere, Finland.
Email: simo.puntanen@uta.fi </ref>


University of Tampere, Finland

and

George P. H. Styan
<ref> Department of Mathematics and Statistics,
McGill University, 805 ouest rue Sherbrooke
Street West, Montréal (Québec), Canada H3A 2K6.
Email:  styan@math.mcgill.ca </ref>


McGill University, Montréal, Canada
</center>
<!-- \maketitle 
     \noindent -->
'''Keywords and Phrases:''' Best linear unbiased, BLUE, BLUP, Gauss--Markov Theorem, Generalized inverse, Ordinary least squares, OLSE.

== Introduction ==
In this article we consider the general linear model
(Gauss--Markov model)
\begin{equation*}
\mx y = \mx X \BETA + \EPS,
\quad \text{or in short }
\M = \{ \mx y, \, \mx X \BETA, \, \sigma^2 \mx V \},
\end{equation*}
where
$\mx X$ is a known $n\times p$ model matrix, the
vector $\mx y$ is an observable $n$-dimensional random vector,
$\BETA$
is a $p\times 1$ vector of unknown parameters, and
$\EPS$ is an unobservable vector of random errors
with expectation
$\EE(\EPS ) = \mx 0,$ and
covariance matrix
$\cov( \EPS) = \sigma^2 \mx V,$
where
$\sigma^2 >0$ is an unknown constant. The nonnegative
definite (possibly singular) matrix $\mx V $ is known. In our
considerations $\sigma ^2$ has no role and hence we may put
$\sigma^2=1.$

As regards the notation,
we will use the symbols
$\mx A',$
$\mx A^{-},$
$\mx A^{+},$
$\C(\mx A),$
$\C(\mx A)^{\bot},$
and
$\NS(\mx A)$
to denote, respectively,
the transpose,
a generalized inverse,
the Moore--Penrose inverse,
the column space,
the orthogonal complement of the column space,
and the null space,
of the matrix $\mx A$.
By $(\mx A:\mx B)$ we denote the partitioned matrix with
$\mx A$ and $\mx B$ as submatrices.
By $\mx A^{\bot}$ we denote any matrix satisfying
$\C(\mx A^{\bot}) = \NS(\mx A') = \C(\mx A)^{\bot}.$
Furthermore, we will write
$\mx P_{\mx A} = \mx A\mx A^{+}
= \mx A(\mx A'\mx A)^{-}\mx A'$
to denote the orthogonal projector (with respect to the standard
inner product) onto
$\C(\mx A).$
In particular, we denote
$\mx{H} = \mx P_{\mx X}$ and $ \mx{M} = \mx I_n - \mx H$.
One choice for $\mx X^{\bot}$ is of course the projector
$\mx M$.

Let $\mx K' \BETA$ be a given vector of parametric functions specified
by $\mx K' \in \rz^{q\times p}.$
Our object is to find a (homogeneous) linear estimator $\mx A \mx y$
which would provide an unbiased and in some sense "best" estimator
for $\mx K' \BETA$ under the model $\M.$
However, not all parametric functions have linear unbiased
estimators; those which have are called ''estimable'' parametric functions,
and then there exists a matrix $\mx A$ such
that
\begin{equation*}
\E(\mx{Ay}) = \mx{AX}\BETA = \mx K' \BETA
\quad \text{for all } \BETA \in \rz^p. 
\end{equation*}
Hence
$\mx{K}' \BETA$ is estimable
if and only if
there exists a matrix $\mx A$ such that $\mx{K}' = \mx{A}\mx{X}$, i.e.,
$ \C(\mx K ) \subset \C(\mx X')$.

The ordinary least squares estimator
of $\mx K' \BETA$ is defined
as
$ \OLSE(\mx K' \BETA) = \mx K' \BETAH, $
where $\BETAH$ is any solution to the normal equation
$\mx X' \mx X \BETAH = \mx X' \mx y$; hence
$\BETA = \BETAH$ minimizes $(\mx y - \mx X\BETA)' (\mx y - \mx X\BETA)$
and it can be expressed as $\BETAH = (\mx X' \mx X) ^{-}\mx X' \mx y,$
while $\mx X\BETAH = \mx H \mx y.$
Now the condition $\C(\mx K ) \subset \C(\mx X')$ guarantees that
$\mx K'\BETAH$ is unique, even though $\BETAH$ may not be unique.

== The Best Linear Unbiased Estimator (BLUE) ==
The expectation $\mx X\BETA$ is trivially estimable
and $\mx{Gy}$ is unbiased for $\mx X\BETA$ whenever
$ \mx{G}\mx X = \mx{X}.$
An unbiased linear estimator $\mx{Gy}$
for $\mx X\BETA$ is defined to be
the ''best'' linear unbiased estimator,
$\BLUE$, for $\mx X\BETA$ under $\M$ if
\begin{equation*}
\cov( \mx{G} \mx y) \leq_{ {\rm L}} \cov( \mx{L} \mx y) \quad
\text{for all } \mx{L} \colon
\mx{L}\mx X = \mx{X},
\end{equation*}
where "$\leq_\text{L}$" refers to the Löwner partial ordering.
In other words, $\mx{G} \mx y$ has the smallest covariance matrix
(in the Löwner sense) among all linear unbiased estimators.
We denote the $\BLUE$ of $\mx X\BETA$ as
$ \BLUE(\mx X\BETA) = \mx X \BETAT. $
If $\mx X$ has full column rank, then $\BETA$ is estimable
and an unbiased estimator $\mx A\mx y$ is the $\BLUE$ for $\BETA$ if
$ \mx{AVA}' \leq_{ {\rm L}} \mx{BVB}'
$ for all $\mx{B}$ such that
$ \mx{BX} = \mx{I}_p. $
The Löwner ordering is a very strong ordering implying for example
\begin{gather*}
\var(\betat_i) \le \var(\beta^{*}_i) \,, \quad i = 1,\dotsc,p , 
\tr [\cov(\BETAT)] \le \tr [\cov(\BETA^{*})] , \qquad
\det[\cov(\BETAT)] \le \det[\cov(\BETA^{*})],
\end{gather*}
for any linear unbiased estimator $\BETA^{*}$ of $\BETA$; here
$\var$ refers to the variance and
"det" denotes
the determinant.

The following theorem gives the "Fundamental $\BLUE$ equation";
see, e.g.,
Rao (1967),
Zyskind (1967)
and Puntanen, Styan and Werner (2000).

'''Theorem 1'''. ''Consider the general linear model'' $ \M =\{\mx y,\,\mx X\BETA,\,\mx V\}$.
''Then the estimator'' $\mx{Gy}$
''is the'' $\BLUE$ ''for'' $\mx X\BETA$ ''if and only if'' $\mx G$
''satisfies the equation''
$$ \label{eq: 30jan09-fundablue}
\mx{G}(\mx{X} : \mx{V}\mx{X}^{\bot} ) = (\mx{X} : \mx{0}).
\tag{1}$$
''The corresponding condition for'' $\mx{Ay}$ ''to be the'' $\BLUE$ ''of an estimable parametric function'' $\mx{K}' \BETA$ ''is'' $ \mx{A}(\mx{X} : \mx{V}\mx{X}^{\bot} ) = (\mx{K}' : \mx{0})$.

It is sometimes convenient to express
(1)
in the following form, see
Rao (1971).

'''Theorem 2'''.[Pandora's Box]
''Consider the general linear model'' $ \M =\{\mx y,\,\mx X\BETA,\,\mx V\}$. ''Then the estimator'' $\mx{Gy}$ ''is the'' $\BLUE$ ''for'' $\mx X\BETA$ ''if and only if there exists a matrix'' $\mx{L} \in \rz^{p \times n}$ ''so that'' $\mx G$ ''is a solution to''
\begin{equation*}
\begin{pmatrix}
\mx V & \mx X \\
\mx X' & \mx 0
\end{pmatrix}
\begin{pmatrix}
\mx G' \\
\mx{L}
\end{pmatrix} =
\begin{pmatrix}
\mx 0 \\
\mx X'
\end{pmatrix}.
\end{equation*}

The equation (1) has a unique solution
for $\mx G$ if and only if $\C(\mx X : \mx V) = \rz^n.$
Notice that under $\M$ we assume that the observed value of
$\mx y$ belongs to the subspace $\C(\mx X : \mx V)$
with probability $1$; this is the consistency condition
of the linear model,
see, e.g.,
Baksalary, Rao and Markiewicz (1992).
The consistency condition means, for example, that whenever we have
some statements which involve the random vector $\mx y$, these
statements need hold only for those values of $\mx y$ that belong
to $\C(\mx{X}:\mx{V}).$
The general solution for $\mx G$
can be expressed, for example, in
the following ways:
\[
\mx G_1 = \mx{X}(\mx{X}'\mx{W}^{-}\mx{X})^{-}\mx{X}'\mx{W}^{-}
+ \mx F_{1}(\mx{I }_n - \mx W\mx W^{-} ) ,
\]
\[
\mx G_2 = \mx{H} - \mx{HVM}(\mx{MVM})^{-}\mx{M} + \mx F_{2}[\mx{I}_n -
\mx{MVM}( \mx{MVM} )^{-} ]\mx M ,
\]
where $\mx F_{1}$ and $\mx F_{2}$ are arbitrary
matrices,
$\mx {W}= \mx V + \mx X\mx U\mx X'$ and $\mx U$ is any arbitrary conformable
matrix such that $\C(\mx W) = \C(\mx X : \mx V).$
Notice that even though $\mx G$ may not be unique, the numerical value
of $\mx G\mx y$ is unique because $\mx y \in \C(\mx X : \mx V).$
If $\mx V$ is positive definite,
then
$\BLUE(\mx X\BETA) = \mx X(\mx X' \mx V^{-1} \mx X)^{-} \mx X' \mx V^{-1} \mx y.$
Clearly $\OLSE(\mx X\BETA) = \mx H\mx y$ is the $\BLUE$ under
$\{ \mx y, \, \mx X\BETA , \, \sigma^2\mx I \}.$
It is also worth noting that the matrix $\mx G$ satisfying
(1) can be interpreted as a
projector: it is a projector onto $\C(\mx X)$ along $\C(\mx V\mx X^{\bot}),$
see Rao (1974).

== OLSE vs. BLUE ==
Characterizing the equality of the Ordinary Least Squares Estimator
$(\OLSE)$ and the $\BLUE$ has received a lot
of attention in the literature,
since Anderson (1948),
but the major breakthroughs were made
by
Rao (1967) and
Zyskind (1967);
for a detailed review, see
Puntanen and Styan (1989).
For some further references from those years we may mention
Kruskal (1968),
Watson (1967),
and Zyskind and Martin (1969).

We present below six characterizations for the $\OLSE$ and
the $\BLUE$ to be equal (with probability $1$).

'''Theorem 3'''.[$\OLSE$ vs. $\BLUE$] ''Consider the general linear model'' $ \M =\{\mx y,\,\mx X\BETA,\,\mx V\}$.
''Then'' $\OLSE(\mx{X}\BETA) = \BLUE(\mx{X}\BETA)$ ''if and only if any one of the following six equivalent conditions holds. (Note:'' $\mx{V}$ ''may be replaced by its Moore--Penrose inverse''
$\mx{V}^+$ ''and'' $\mx{H}$ ''and'' $\mx{M} = \mx I_n - \mx H$ ''may be interchanged.)''
:(1) $\mx{HV} = \mx{VH} $, 
:(2) $ \mx{H}\mx{V}\mx{M} = \mx{0} $, 
:(3) $ \C(\mx{V}\mx{H})\subset\C(\mx H) $,
:(4) $ \C(\mx{X}) $ ''has a basis comprising'' $ r= \rank(\mx X) $ ''orthonormal eigenvectors of'' $ \mx V $, 
:(5) $ \mx{V} = \mx{HAH} + \mx{MBM} $  ''for some'' $ \mx A $ ''and'' $ \mx B $, 
:(6) $ \mx{V} = \alpha\mx{I}_n + \mx{HKH} + \mx{M}\mx L\mx M $ ''for some'' $ \alpha \in \rz $, ''and'' $ \mx K $ ''and'' $\mx L$. 

Theorem 3 shows at once that
under $\{ \mx y, \, \mx X\BETA, \, \mx I_n \}$ the $\OLSE$ of
$\mx X\BETA$ is trivially the $\BLUE$; this result is often called
the Gauss--Markov Theorem.

== Two Linear Models ==
Consider now two linear models
$ \M_{1} = \{ \mx y, \, \mx X\BETA, \, \mx V_1 \}$
and $ \M_{2} = \{ \mx y, \, \mx X\BETA, \, \mx V_2 \} $,
which differ only in their covariance matrices.
For the proof of the
following proposition and related discussion, see, e.g.,
Rao (1971, Th. 5.2, Th. 5.5),
and Mitra and Moore (1973, Th. 3.3, Th. 4.1--4.2).

'''Theorem 4'''.
''Consider the linear models''
$ \M_1 = \{\mx y, \, \mx X\BETA, \, \mx V_1 \}$
''and''
$ \M_2 = \{ \mx y, \, \mx X\BETA, \, \mx V_2 \},$
''and let the notation''
$ \{\BLUE(\mx X\BETA \mid \M_1) \} \subset \{\BLUE(\mx X\BETA \mid \M_2) \} $
''mean that every representation of the'' $\BLUE$ ''for'' $\mx X\BETA$ ''under'' $\M_1$
''remains the'' $\BLUE$ ''for'' $\mx X\BETA$ ''under'' $\M_2$. ''Then the following statements are equivalent:''
:(1) $ \{ \BLUE(\mx X\BETA \mid \M_1) \} \subset
\{ \BLUE(\mx X\BETA \mid \M_2) \} $,
:(2) $ \C(\mx V_2\mx X^{\bot}) \subset \C(\mx V_1 \mx X^\bot) $,
:(3) $ \mx V_2 = \mx V_1+ \mx{X} \mx N_1 \mx X' +
\mx V_1\mx M\mx N_2 \mx M \mx V_1 $, ''for some'' $ \mx N_1 $ ''and'' $ \mx N_2 $,  
:(4) $ \mx V_2 =
\mx{X} \mx N_3 \mx X' + \mx V_1\mx M\mx N_4 \mx M \mx V_1 $, ''for some'' $ \mx N_3 $ ''and'' $ \mx N_4 $.

Notice that obviously
\begin{align*}
\{ \BLUE(\mx X \BETA \mid \M_1) \}
=
\{ \BLUE(\mx X \BETA \mid \M_2) \}
\iff
\C(\mx V_2\mx X^{\bot}) = \C(\mx V_1 \mx X^\bot).
\end{align*}
For the equality
between the
$\BLUE$s of $\mx X_1\BETA_1$
under two partitioned models, see
Haslett and Puntanen (2010a).

== Model with New Observations: Best Linear Unbiased Predictor (BLUP) ==
Consider the model
$ \M = \{\mx y,\,\mx X\BETA,\,\mx V\},$
and
let $\mx y_f$
denote an $m\times 1$ unobservable random vector containing
''new observations''. The new observations are assumed to follow
the linear model
$ \mx y_f = \mx X_f\BETA +\EPS_f ,$
where $\mx X_f$ is a known $m\times p$ model matrix associated with new
observations, $\BETA$ is the same vector of unknown parameters as
in $\M$, and $\EPS_f$ is an $m \times 1$ random
error vector associated with new observations.
Our goal is to predict the random vector $\mx y_f$
on the basis of $\mx y$.
The expectation and the covariance matrix are
\begin{equation*}
\E\begin{pmatrix}
\mx y \\
\mx y_f
\end{pmatrix} =
\begin{pmatrix}
\mx X\BETA \\
\mx X_f\BETA
\end{pmatrix} , \quad
\cov\begin{pmatrix}
\mx y \\
\mx y_f
\end{pmatrix}
=
\begin{pmatrix}
\mx V = \mx V_{11} & \mx{V}_{12} \\
\mx{V}_{21} & \mx V_{22}
\end{pmatrix} ,
\end{equation*}
which we may write as
\begin{equation*}
\M_f = \left \{
\begin{pmatrix}
\mx y \\
\mx y_f
\end{pmatrix},\,
\begin{pmatrix}
\mx X\BETA \\
\mx X _f\BETA
\end{pmatrix},\,
\begin{pmatrix}
\mx V & \mx{V}_{12} \\
\mx{V}_{21} & \mx V_{22}
\end{pmatrix} \right \}.
\end{equation*}

A linear predictor $\mx{Ay}$ is said to be unbiased for $\mx y_f$ if
$ \E(\mx{Ay}) = \E(\mx{y}_f) = \mx X_f\BETA$
for all $\BETA\in\rz^{p}.$
Then the random vector
$\mx y_f$ is said to be unbiasedly predictable.
Now an unbiased linear predictor $\mx{Ay}$ is the
best linear unbiased predictor, $\BLUP$, for $\mx y_f$
if the Löwner ordering
\begin{equation*}
\cov(\mx{Ay}-\mx y_f) \leq_{ {\rm L}} \cov(\mx{By}-\mx y_f)
\end{equation*}
holds for all $\mx B$ such that $\mx{By}$ is an unbiased linear
predictor for $\mx{y}_f$.

The following theorem characterizes the $\BLUP$;
see, e.g.,
Christensen (2002, p. 283),
and
Isotalo and Puntanen (2006, p. 1015).

'''Theorem 5''' (Fundamental $\BLUP$ equation)
''Consider the linear model''
$\M_f$, ''where''
$\mx{X}_f\BETA$ ''is a given estimable parametric function. Then the linear estimator'' $\mx{Ay}$
''is the best linear unbiased predictor'' ($\BLUP$) ''for'' $\mx y_f$
''if and only if'' $\mx{A}$ ''satisfies the equation''
\begin{equation*}
\mx{A}(\mx{X} : \mx{V} \mx X^{\bot}) = (\mx X_f : \mx{V}_{21} \mx X^{\bot} ).
\end{equation*}
''In terms of Pandora's Box (Theorem 2),'' $\mx{Ay}$ ''is the'' $\BLUP$
''for'' $\mx y_f$ ''if and only if there exists a matrix'' $\mx L$ ''such that''
$\mx{A}$ ''satisfies the equation''
\begin{equation*}
\begin{pmatrix}
\mx V & \mx X \\
\mx X' & \mx 0
\end{pmatrix}
\begin{pmatrix}
\mx A' \\
\mx L
\end{pmatrix} =
\begin{pmatrix}
\mx{V}_{12} \\
\mx X_{f}'
\end{pmatrix}.
\end{equation*}

== The Mixed Model ==
A mixed linear model can be presented as
\begin{equation*}
\mx y = \mx X\BETA + \mx Z \GAMMA +\EPS ,
\quad \text{or shortly } \quad
\M_{\mathrm{mix}}
= \{ \mx y,\, \mx X\BETA + \mx Z\GAMMA, \, \mx D,\,\mx R \} ,
\end{equation*}
where $\mx X \in \rz^{n \times p}$ and $\mx Z \in \rz^{n \times q}$ are
known matrices, $\BETA \in \rz^{p}$ is a vector of unknown fixed
effects, $\GAMMA$ is an unobservable vector ($q$ elements) of
''random effects'' with
$\cov(\GAMMA,\EPS) = \mx 0_{q \times p}$ and
\begin{equation*}
\E(\GAMMA) = \mx 0_q , \quad
\cov(\GAMMA) = \mx D_{q \times q}, \quad
\E(\EPS) = \mx 0_n \,, \quad
\cov(\EPS) = \mx R_{n\times n}.
\end{equation*}
This leads directly to:

'''Theorem 6'''.
''Consider the mixed model''
$ \M_{\mathrm{mix}}
= \{ \mx y,\, \mx X\BETA + \mx Z\GAMMA, \, \mx D,\,\mx R \}.$
''Then the linear estimator''
$\mx B \mx y$ ''is the'' $\BLUE$ ''for'' $\mx X\BETA$ ''if and only if''
\begin{equation*}
\mx B(\mx X : \SIGMA \mx X^{\bot}) = (\mx X : \mx{0}) ,
\end{equation*}
''where'' $\SIGMA= \mx Z\mx D\mx Z' + \mx R$.
''Moreover,''
$\mx A \mx y$ ''is the'' $\BLUP$ ''for'' $\GAMMA$ ''if and only if''
\begin{equation*}
\mx A(\mx X : \SIGMA \mx X^{\bot}) = (\mx 0 : \mx{D}\mx{Z}' \mx X^{\bot}).
\end{equation*}
''In terms of Pandora's Box (Theorem 2),'' $\mx A \mx y = \BLUP(\GAMMA)$
''if and only if there exists a matrix'' $\mx L$ ''such that'' $\mx{A}$ ''satisfies the equation''
\begin{equation*}
\begin{pmatrix}
\SIGMA & \mx X \\
\mx X' & \mx 0
\end{pmatrix}
\begin{pmatrix}
\mx A' \\
\mx L
\end{pmatrix} =
\begin{pmatrix}
\mx Z \mx D \\
\mx 0
\end{pmatrix}.
\end{equation*}

For the equality
between the
$\BLUP$s
under two mixed models, see
Haslett and Puntanen (2010b, 2010c).

=== Note ===
Reprinted with permission from Lovric, Miodrag (2011),
''International
Encyclopedia of Statistical Science.'' Heidelberg:
Springer Science+Business Media, LLC.


====References====
{|
|-
|valign="top"|{{Ref|1}}||valign="top"|  Anderson, T. W. (1948). On the theory of testing serial correlation. ''Skandinavisk Aktuarietidskrift'', '''31''', 88--116.
|-
|valign="top"|{{Ref|2}}||valign="top"|  Baksalary, Jerzy K.; Rao, C. Radhakrishna and Markiewicz, Augustyn (1992). A study of the influence of the `natural restrictions' on estimation problems in the singular Gauss--Markov model, ''Journal of Statistical Planning and Inference'', '''31''', 335--351.
|-
|valign="top"|{{Ref|3}}||valign="top"|  Christensen, Ronald (2002). ''Plane Answers to Complex Questions: The Theory of Linear Models,'' 3rd Edition. Springer, New York.
|-
|valign="top"|{{Ref|4}}||valign="top"|  Haslett, Stephen J. and Puntanen, Simo (2010a). Effect of adding regressors on the equality of the BLUEs under two linear models. ''Journal of Statistical Planning and Inference'', '''140''', 104--110,
|-
|valign="top"|{{Ref|5}}||valign="top"|  Haslett, Stephen J. and Puntanen, Simo (2010b). Equality of BLUEs or BLUPs under two linear models using stochastic restrictions. ''Statistical Papers'', '''51''', 465--475.
|-
|valign="top"|{{Ref|6}}||valign="top"|  Haslett, Stephen J. and Puntanen, Simo (2010c). On the equality of the BLUPs under two linear mixed models. ''Metrika'', available online, DOI 10.1007/s00184-010-0308-6.
|-
|valign="top"|{{Ref|7}}||valign="top"|  Isotalo, Jarkko and Puntanen, Simo (2006). Linear prediction sufficiency for new observations in the general Gauss--Markov model. ''Communications in Statistics: Theory and Methods'', '''35''', 1011--1023.
|-
|valign="top"|{{Ref|8}}||valign="top"|  Kruskal, William (1967). When are Gauss--Markov and least squares estimators identical? {A} coordinate-free approach. ''The Annals of Mathematical Statistics'', '''39''', 70--75.
|-
|valign="top"|{{Ref|9}}||valign="top"|  Mitra, Sujit Kumar and Moore, Betty Jeanne (1973). Gauss--Markov estimation with an incorrect dispersion matrix. ''Sankhya, Series A'', '''35''', 139--152.
|-
|valign="top"|{{Ref|10}}||valign="top"|  Puntanen, Simo and Styan, George P. H. (1989). The equality of the ordinary least squares estimator and the best linear unbiased estimator [with comments by Oscar Kempthorne and by Shayle R. Searle and with "Reply" by the authors]. ''The American Statistician'', '''43''', 153--164.
|-
|valign="top"|{{Ref|11}}||valign="top"|  Puntanen, Simo; Styan, George P. H. and Werner, Hans Joachim (2000). Two matrix-based proofs that the linear estimator ''Gy'' is the best linear unbiased estimator. ''Journal of Statistical Planning and Inference'', '''88''', 173--179.
|-
|valign="top"|{{Ref|12}}||valign="top"|  Rao, C. Radhakrishna (1967). Least squares theory using an estimated dispersion matrix and its application to measurement of signals. In ''Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability: Berkeley, California, 1965/1966'', vol. 1 (Eds. Lucien M. Le Cam & Jerzy Neyman), University of California Press, Berkeley, pp. 355--372.
|-
|valign="top"|{{Ref|13}}||valign="top"|  Rao, C. Radhakrishna (1971). Unified theory of linear estimation. ''Sankhya, Series A'', '''33''', 371--394. [Corrigenda (1972), '''34''', p. 194 and p. 477.]
|-
|valign="top"|{{Ref|14}}||valign="top"|  Rao, C. Radhakrishna (1974). Projectors, generalized inverses and the BLUE's. ''Journal of the Royal Statistical Society, Series B'', '''36''', 442--448.
|-
|valign="top"|{{Ref|15}}||valign="top"|  Watson, Geoffrey S. (1967). Linear least squares regression. ''The Annals of Mathematical Statistics'', '''38''', 1679--1699.
|-
|valign="top"|{{Ref|16}}||valign="top"|  Zyskind, George (1967). On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models. ''The Annals of Mathematical Statistics'', '''38''', 1092--1109.
|-
|valign="top"|{{Ref|17}}||valign="top"|  Zyskind, George and Martin, Frank B. (1969). On best linear estimation and general Gauss--Markov theorem in linear models with arbitrary nonnegative covariance structure. ''SIAM Journal on Applied Mathematics'', '''17''', 1190--1202.
|}


<!-- \end{document} -->

<references />

[[Category:Statprob]]