View source for Errors, theory of

The branch of [[Mathematical statistics|mathematical statistics]] devoted to the inference of accurate conclusions about the numerical values of approximately measured quantities, as well as on the errors in the measurements. Repeated measurements of one and the same constant quantity generally give different results, since every measurement contains a certain error. There are three basic types of error: systematic, gross and random. Systematic errors always either overestimate or underestimate the results of measurements and arise for specific reasons (incorrect set-up of measuring equipment, the effect of environment, etc.), which systematically affect the measurements and alter them in one direction. The estimation of systematic errors is achieved using methods which go beyond the confines of mathematical statistics (see [[Processing of observations|Processing of observations]]). Gross errors (often called outliers) arise from miscalculations, incorrect reading of the measuring equipment, etc. The results of measurements which contain gross errors differ greatly from other results of measurements and are therefore often easy to identify. Random errors arise from various reasons which have an unforeseen effect on each of the measurements, both in overestimating and in underestimating results.

The theory of errors is only concerned with the study of gross and random errors. The basic problems of the theory of errors are to study the distribution laws of random errors, to seek estimates (see [[Statistical estimator|Statistical estimator]]) of unknown parameters using the results of measurements, to establish the errors in these estimates, and to identify gross errors. Let the values <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362401.png" /> be obtained as a result of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362402.png" /> independent, equally accurate measurements of a certain unknown variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362403.png" />. The differences

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362404.png" /></td> </tr></table>

are called the true errors. In terms of the probability theory of errors all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362405.png" /> are treated as random variables; independence of measurements is understood to be mutual independence of the random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362406.png" />. The equal accuracy of the measurements is treated broadly as an identical distribution: The true errors of equally accurate measurements are identically distributed random variables. The [[Mathematical expectation|mathematical expectation]] of the true errors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362407.png" /> is then called the systematic error, while the differences <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362408.png" /> are called the random errors. Thus, the absence of a systematic error means that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e0362409.png" />, and, in this situation, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624010.png" /> are random errors. The variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624011.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624012.png" /> is the [[Standard deviation|standard deviation]], is called the measure of accuracy (when a systematic error occurs, the measure of accuracy is expressed by the relation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624013.png" />). Equal accuracy of measurements is understood in a narrow sense to mean the equality of the measure of accuracy for all the results of the measurements. The incidence of gross errors signifies the disruption of equal accuracy (both in the broad and narrow sense) for certain specific measurements. As an estimator of the unknown value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624014.png" /> one usually takes the arithmetic mean from the results of the measurements:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624015.png" /></td> </tr></table>

while the differences <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624016.png" /> are called the apparent errors. The choice of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624017.png" /> as an estimator for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624018.png" /> is based on the fact that for any sufficiently large number <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624019.png" /> of equally accurate measurements with no systematic error, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624020.png" />, with probability arbitrarily close to one, differs by an arbitrarily small amount from the unknown variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624021.png" /> (see [[Law of large numbers|Law of large numbers]]); <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624022.png" /> is free of systematic errors (estimators with this property are called unbiased, cf. also [[Unbiased estimator|Unbiased estimator]]), and its variance is

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624023.png" /></td> </tr></table>

Experience has shown that in practice the random errors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624024.png" /> are very often subject to almost normal distributions (the reasons for this are revealed in the so-called [[Limit theorems|limit theorems]] of probability theory). In this case the variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624025.png" /> has an almost normal distribution with mathematical expectation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624026.png" /> and variance <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624027.png" />. If the distributions of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624028.png" /> are exactly normal, then the variance of every other unbiased estimator for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624029.png" />, for example the median (cf. [[Median (in statistics)|Median (in statistics)]]), is not less than <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624030.png" />. If the distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624031.png" /> is not normal, then the latter property need not hold (see the example under [[Rao–Cramér inequality|Rao–Cramér inequality]]).

If the variance <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624032.png" /> of separate measurements is previously unknown, then the variable

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624033.png" /></td> </tr></table>

is used as estimator for it (<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624034.png" />, i.e. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624035.png" /> is an unbiased estimator for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624036.png" />). If the random errors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624037.png" /> have a normal distribution, then the relation

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624038.png" /></td> </tr></table>

is subject to the [[Student distribution|Student distribution]] with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624039.png" /> degrees of freedom. This can be used to estimate the error of the approximate equality <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624040.png" /> (see [[Least squares, method of|Least squares, method of]]).

The variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624041.png" /> has under the same assumptions a [[Chi-squared distribution| "chi-squared"  distribution]] with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624042.png" /> degrees of freedom. This enables one to estimate the error of the approximate equality <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624043.png" />. It can be demonstrated that the relative error will not exceed the number <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624044.png" /> with probability

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624045.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624046.png" /> is the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624047.png" />-distribution function,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624048.png" /></td> </tr></table>

If certain measurements contain gross errors, then the above rules for estimating <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624049.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624050.png" /> will give distorted results. It is therefore very important to be able to differentiate measurements which contain gross errors from those which are subject only to random errors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624051.png" />. For the case where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624052.png" /> are independent and have an identical normal distribution, a comprehensive method for identifying measurements which contain gross errors was proposed by N.V. Smirnov [[#References|[3]]].

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  Yu.V. Linnik,   "Methode der kleinste Quadraten in moderner Darstellung" , Deutsch. Verlag Wissenschaft.  (1961)  (Translated from Russian)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  L.N. Bol'shev,   N.V. Smirnov,   "Tables of mathematical statistics" , ''Libr. math. tables'' , '''46''' , Nauka  (1983)  (In Russian)  (Processed by L.S. Bark and E.S. Kedrova)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  N.V. Smirnov,   "On the estimation of the maximum term in a series of observations"  ''Dokl. Akad. Nauk SSSR'' , '''33''' :  5  (1941)  pp. 346–349  (In Russian)</TD></TR></table>



====Comments====
Modern developments in the treatment of errors include robust estimation and outlier detection and treatment.

The intuitive definition of an outlier is: an observation which deviates so much from other observations as to arouse suspicions that it was generated by a different mechanism (than the one under observation). This includes errors such as can arise for instance because data were copied incorrectly. Outliers are dangerous for many statistical procedures. One way to deal with outliers is to use outlier tests to accept or reject the hypothesis that an observation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624053.png" /> belongs to a sample from a random variable or not. The observations for which this hypothesis is rejected are then removed. Other methods for dealing with outliers include censoring, the use of robust methods and Winsorization (which is robust), of which the underlying idea is to move all excessively outlying observations in some systematic way to a position near the more central observations. One mechanism which may cause (apparent) outliers is when the data come from a heavy tailed distribution. Another one is when the data come from two distributions: a basic one which yields  "good"  observations and a second contaminating distribution. Which treatment of outliers is appropriate depends of course heavily on the mechanism generating the outliers. A selection of references on outliers and methods for treating them is [[#References|[a1]]]–.

Robust statistics in a loose sense tries to deal with the fact that many often made assumptions such as normality, linearity, independence<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624054.png" /> are at best approximations to the real situation. Thus, one looks for tests, statistical procedures<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624055.png" /> which are, for instance, insensitive to the assumption that the underlying distribution is normal. Let the statistical model being used be, e.g., a parametrized family of distributions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624056.png" /> conceived of as lying in a larger space of distributions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624057.png" />. One main aspect of robust statistics is then the study of the effects of deformations of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624058.png" /> in <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/e/e036/e036240/e03624059.png" /> on the various statistical procedures being used. Similar concerns have motivated the study of deformations in other parts of mathematics, for instance deformations of dynamical systems. More generally, robust statistics is concerned with statistical concepts which describe the behaviour of statistical procedures not only under parametric models but also in the neighbourhoods of such models.

Robust statistics is by now is large active field. A selection of books on the topic is [[#References|[a5]]]–[[#References|[a7]]].

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  Th.S. Ferguson,   "Rules for rejection of outliers"  ''Rev. Inst. Int. Stat.'' , '''29'''  (1961)  pp. 29–43</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  D.M. Hawkins,   "Identification of outliers" , Chapman &amp; Hall  (1980)</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  W.J. Dixon,   "Simplified estimation from censored normal samples"  ''Ann. Math. Stat.'' , '''31'''  (1960)  pp. 385–391</TD></TR><TR><TD valign="top">[a4a]</TD> <TD valign="top">  A.E. Sarhan,   B.G. Greenberg,   "Estimation of location and scale parameters by order statistics from singly and doubly censored samples I"  ''Ann. Math. Stat.'' , '''27'''  (1956)  pp. 427–451</TD></TR><TR><TD valign="top">[a4b]</TD> <TD valign="top">  A.E. Sarhan,   B.G. Greenberg,   "Estimation of location and scale parameters by order statistics from singly and doubly censored samples II"  ''Ann. Math. Stat.'' , '''29'''  (1958)  pp. 79–105</TD></TR><TR><TD valign="top">[a5]</TD> <TD valign="top">  P.J. Huber,   "Robust statistics" , Wiley  (1981)</TD></TR><TR><TD valign="top">[a6]</TD> <TD valign="top">  F.R. Hampel,   E.M. Ronchetti,   P.J. Rousseeuw,   W.A. Stahel,   "Robust statistics. The approach based on influence functions" , Wiley  (1986)</TD></TR><TR><TD valign="top">[a7]</TD> <TD valign="top">  W.J.J. Rey,   "Introduction to robust and quasi-robust statistical methods" , Springer  (1983)</TD></TR><TR><TD valign="top">[a8]</TD> <TD valign="top">  W.T. Federer,   "Statistics and society. Data collection and interpretation" , M. Dekker  (1973)</TD></TR></table>