Biased estimator

A statistical estimator whose expectation does not coincide with the value being estimated.

Let $X$ be a random variable taking values in a sampling space $( \mathfrak X , {\mathcal B} , {\mathsf P} _ \theta )$, $\theta \in \Theta$, and let $T = T (X)$ be a statistical point estimator of a function $f ( \theta )$ defined on the parameter set $\Theta$. It is assumed that the mathematical expectation ${\mathsf E} _ \theta \{ T \}$ of $T$ exists. If the function

$$b ( \theta ) = \ {\mathsf E} _ \theta \{ T \} - f ( \theta ) = \ {\mathsf E} _ \theta \{ T - f ( \theta ) \}$$

is not identically equal to zero, that is, $b ( \theta ) \not\equiv 0$, then $T$ is called a biased estimator of $f ( \theta )$ and $b ( \theta )$ is called the bias or systematic error of $T$.

Example. Let $X _ {1} \dots X _ {n}$ be mutually-independent random variables with the same normal distribution $N _ {1} (a, \sigma ^ {2} )$, and let

$$\overline{X}\; = \ { \frac{X _ {1} + \dots + X _ {n} }{n} } .$$

Then the statistic

$$S _ {n} ^ {2} = \ { \frac{1}{n} } \sum _ {i = 1 } ^ { n } (X _ {i} - \overline{X}\; ) ^ {2}$$

is a biased estimator of the variance $\sigma ^ {2}$ since

$${\mathsf E} \{ S _ {n} ^ {2} \} = \ { \frac{n - 1 }{n} } \sigma ^ {2} = \ \sigma ^ {2} - { \frac{\sigma ^ {2} }{n} } ,$$

that is, the estimator $S _ {n} ^ {2}$ has bias $b ( \sigma ^ {2} ) = - \sigma ^ {2} /n$. The mean-square error of this biased estimator is

$${\mathsf E} \{ (S _ {n} ^ {2} - \sigma ^ {2} ) ^ {2} \} = \ \frac{2n - 1 }{n ^ {2} } \sigma ^ {4} .$$

The best unbiased estimator of $\sigma ^ {2}$ is the statistic

$$s _ {n} ^ {2} = \ { \frac{n}{n - 1 } } S _ {n} ^ {2} = \ { \frac{1}{n - 1 } } \sum _ {i = 1 } ^ { n } (X _ {i} - \overline{X}\; ) ^ {2} ,$$

with mean-square error

$${\mathsf D} \{ s _ {n} ^ {2} \} = \ {\mathsf E} \{ (s _ {n} ^ {2} - \sigma ^ {2} ) ^ {2} \} = \ { \frac{2}{n - 1 } } \sigma ^ {4} .$$

When $n > 2$, the mean-square error of the biased estimator $S _ {n} ^ {2}$ is less than that of the best unbiased estimator $s _ {n} ^ {2}$.

There are situations when unbiased estimators do not exist. For example, there is no unbiased estimator for the absolute value $| a |$ of the mathematical expectation $a$ of the normal law $N _ {1} (a, \sigma ^ {2} )$, that is, it is only possible to construct biased estimators for $| a |$.

References

 [1] H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
How to Cite This Entry:
Biased estimator. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Biased_estimator&oldid=46049
This article was adapted from an original article by M.S. Nikulin (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article