Biased estimator

A statistical estimator whose expectation does not coincide with the value being estimated.

Let $ X $ be a random variable taking values in a sampling space $ ( \mathfrak X , {\mathcal B} , {\mathsf P} _ \theta ) $, $ \theta \in \Theta $, and let $ T = T (X) $ be a statistical point estimator of a function $ f ( \theta ) $ defined on the parameter set $ \Theta $. It is assumed that the mathematical expectation $ {\mathsf E} _ \theta \{ T \} $ of $ T $ exists. If the function

$$ b ( \theta ) = \ {\mathsf E} _ \theta \{ T \} - f ( \theta ) = \ {\mathsf E} _ \theta \{ T - f ( \theta ) \} $$

is not identically equal to zero, that is, $ b ( \theta ) \not\equiv 0 $, then $ T $ is called a biased estimator of $ f ( \theta ) $ and $ b ( \theta ) $ is called the bias or systematic error of $ T $.

Example. Let $ X _ {1} \dots X _ {n} $ be mutually-independent random variables with the same normal distribution $ N _ {1} (a, \sigma ^ {2} ) $, and let

$$ \overline{X}\; = \ { \frac{X _ {1} + \dots + X _ {n} }{n} } . $$

Then the statistic

$$ S _ {n} ^ {2} = \ { \frac{1}{n} } \sum _ {i = 1 } ^ { n } (X _ {i} - \overline{X}\; ) ^ {2} $$

is a biased estimator of the variance $ \sigma ^ {2} $ since

$$ {\mathsf E} \{ S _ {n} ^ {2} \} = \ { \frac{n - 1 }{n} } \sigma ^ {2} = \ \sigma ^ {2} - { \frac{\sigma ^ {2} }{n} } , $$

that is, the estimator $ S _ {n} ^ {2} $ has bias $ b ( \sigma ^ {2} ) = - \sigma ^ {2} /n $. The mean-square error of this biased estimator is

$$ {\mathsf E} \{ (S _ {n} ^ {2} - \sigma ^ {2} ) ^ {2} \} = \ \frac{2n - 1 }{n ^ {2} } \sigma ^ {4} . $$

The best unbiased estimator of $ \sigma ^ {2} $ is the statistic

$$ s _ {n} ^ {2} = \ { \frac{n}{n - 1 } } S _ {n} ^ {2} = \ { \frac{1}{n - 1 } } \sum _ {i = 1 } ^ { n } (X _ {i} - \overline{X}\; ) ^ {2} , $$

with mean-square error

$$ {\mathsf D} \{ s _ {n} ^ {2} \} = \ {\mathsf E} \{ (s _ {n} ^ {2} - \sigma ^ {2} ) ^ {2} \} = \ { \frac{2}{n - 1 } } \sigma ^ {4} . $$

When $ n > 2 $, the mean-square error of the biased estimator $ S _ {n} ^ {2} $ is less than that of the best unbiased estimator $ s _ {n} ^ {2} $.

There are situations when unbiased estimators do not exist. For example, there is no unbiased estimator for the absolute value $ | a | $ of the mathematical expectation $ a $ of the normal law $ N _ {1} (a, \sigma ^ {2} ) $, that is, it is only possible to construct biased estimators for $ | a | $.

References

[1]	H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)