View source for Robust statistics

The branch of [[Mathematical statistics|mathematical statistics]] concerned with the construction and investigation of statistical procedures (such as parameter estimators and tests) that still behave well when the usual assumptions are not satisfied. For instance, the observations may not follow the normal (Gaussian) distribution exactly, because in practice the data often contain outliers (aberrant values) which are caused by transcription errors, misplaced decimal points, exceptional phenomena, etc.

It has been known for many years that the classical procedures (such as the [[Sample average|sample average]]) loose their optimality, and can in fact give arbitrarily bad results, when even a small percentage of the data are outlying. In such situations applied statisticians have turned to alternative methods (e.g. the [[Sample median|sample median]]) which they knew to be less affected by possible outlying values. However, the intuitive notion of robustness was not formalized until the 1960's. The theoretical foundations of robust statistics have been developed in the three stages described below.

The first mathematical approach is due to P.J. Huber [[#References|[a1]]], who found the solution <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825301.png" /> to a minimax variational problem:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825302.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825303.png" /> denotes the asymptotic variance of an estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825304.png" /> at a distribution <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825305.png" />. The set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825306.png" /> (for some <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825307.png" />) consists of all mixture distributions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825308.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r0825309.png" /> is the standard [[Normal distribution|normal distribution]] and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253010.png" /> is arbitrary (e.g., the Cauchy distribution). In the framework of a univariate location parameter, the solution <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253011.png" /> has come to be known as the Huber estimator. For a survey of this work see [[#References|[a2]]].

The second approach was based on the influence function [[#References|[a3]]]

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253012.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253013.png" /> is the probability distribution which puts all its mass in the point <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253014.png" />. The influence function describes the effect of a single outlier at <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253015.png" /> on (the asymptotic version of) the estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253016.png" />. In this approach one obtains optimally robust estimators and tests by maximizing the asymptotic efficiency subject to an upper bound on the supremum norm of the influence function. Unlike the minimax variance approach, it can also be applied to multivariate problems. The main results of this methodology are found in [[#References|[a4]]].

The third approach aims for a high breakdown point. The breakdown point of an estimator is the largest fraction of arbitrary outliers it can tolerate without becoming unbounded. This goes beyond the second approach, where only the effect of a single outlier was considered. In [[Linear regression|linear regression]] analysis, the first equivariant high-breakdown estimator was the least median of squares [[#References|[a5]]], defined by

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253017.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253018.png" /> is the vector of regression coefficients and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253019.png" /> is the residual of the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/r/r082/r082530/r08253020.png" />-th observation. High-breakdown estimators also exist for other multivariate situations. A survey of these methods is given in [[#References|[a6]]], which also describes applications.

At present (1990's), research is focussed on the construction of estimators that have a high breakdown point as well as bounded influence function and good asymptotic efficiency. The more recently developed estimators make increasing use of computing resources.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  P.J. Huber,   "Robust estimation of a location parameter"  ''Ann. Math. Stat.'' , '''35'''  (1964)  pp. 73–101</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  P.J. Huber,   "Robust statistics" , Wiley  (1981)</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  F.R. Hampel,   "The influence curve and its role in robust estimation"  ''J. Amer. Statist. Assoc.'' , '''69'''  (1974)  pp. 383–393</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  F.R. Hampel,   E.M. Ronchetti,   P.J. Rousseeuw,   W.A. Stahel,   "Robust statistics. The approach based on influence functions" , Wiley  (1986)</TD></TR><TR><TD valign="top">[a5]</TD> <TD valign="top">  P.J. Rousseeuw,   "Least median of squares regression"  ''J. Amer. Statist. Assoc.'' , '''79'''  (1984)  pp. 871–880</TD></TR><TR><TD valign="top">[a6]</TD> <TD valign="top">  P.J. Rousseeuw,   A.M. Leroy,   "Robust regression and outlier detection" , Wiley  (1987)</TD></TR></table>