View source for Significance level

''of a statistical test''

The probability of incorrectly rejecting the basic hypothesis being tested, when it is valid. In the theory of statistical hypotheses testing (cf. [[Statistical hypotheses, verification of|Statistical hypotheses, verification of]]), the significance level is also called the probability of an error of the first kind. The concept first arose in connection with the problem of testing for compatibility of a theory with experimental data. For example, suppose that observations are being conducted on the values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850701.png" /> random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850702.png" /> and that, on the basis of these data, it is required to test a hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850703.png" />, according to which the joint distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850704.png" /> has some specific property. An appropriate statistical test is constructed with the aid of a suitably selected function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850705.png" />; this function usually assumes small values when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850706.png" /> is true, and large values when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850707.png" /> is false. In particular, if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850708.png" /> are the outcomes of independent measurements (with error) of some known constant <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850709.png" /> and the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507010.png" /> states that no systematic errors are involved, then a reasonable choice of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507011.png" /> is <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507012.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507013.png" /> is the number of measured values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507014.png" /> that exceed the true value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507015.png" />. A large observed value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507016.png" /> may be considered a significant statistical refutation of the hypothetical agreement between the experimental outcome and the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507017.png" />. The corresponding [[Significance test|significance test]] is a rule according to which values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507018.png" /> are considered significant if they exceed a prescribed critical value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507019.png" />. In its turn, the choice of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507020.png" /> is governed by the significance level, which equals the probability of the event <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507021.png" /> in the case that the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507022.png" /> is true.

Selection of a significance level should also take into account the unavoidable errors incurred when any specific significance level is employed. For example, if the significance level is excessively high, the main error will stem from rejection of a true hypothesis; but if the significance level is low, the error will usually arise from accepting a false hypothesis. In practice, the most commonly adopted significance levels in statistical calculations range from <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507023.png" /> to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507024.png" />. Significance levels lower than <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507025.png" /> are used, for example, in statistical detection of toxic medical preparates, and also in other special situations where the overriding purpose is to ensure against incorrect rejection of the hypothesis being tested. See also [[Confidence estimation|Confidence estimation]].

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  H. Cramér,   "Mathematical methods of statistics" , Princeton Univ. Press  (1946)</TD></TR></table>



====Comments====


====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  E.L. Lehmann,   "Testing statistical hypotheses" , Wiley  (1969)</TD></TR></table>