Namespaces
Variants
Actions

Difference between revisions of "Significance level"

From Encyclopedia of Mathematics
Jump to: navigation, search
(Importing text file)
 
(TeX)
 
Line 1: Line 1:
 +
{{TEX|done}}
 
''of a statistical test''
 
''of a statistical test''
  
The probability of incorrectly rejecting the basic hypothesis being tested, when it is valid. In the theory of statistical hypotheses testing (cf. [[Statistical hypotheses, verification of|Statistical hypotheses, verification of]]), the significance level is also called the probability of an error of the first kind. The concept first arose in connection with the problem of testing for compatibility of a theory with experimental data. For example, suppose that observations are being conducted on the values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850701.png" /> random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850702.png" /> and that, on the basis of these data, it is required to test a hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850703.png" />, according to which the joint distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850704.png" /> has some specific property. An appropriate statistical test is constructed with the aid of a suitably selected function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850705.png" />; this function usually assumes small values when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850706.png" /> is true, and large values when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850707.png" /> is false. In particular, if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850708.png" /> are the outcomes of independent measurements (with error) of some known constant <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s0850709.png" /> and the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507010.png" /> states that no systematic errors are involved, then a reasonable choice of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507011.png" /> is <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507012.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507013.png" /> is the number of measured values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507014.png" /> that exceed the true value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507015.png" />. A large observed value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507016.png" /> may be considered a significant statistical refutation of the hypothetical agreement between the experimental outcome and the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507017.png" />. The corresponding [[Significance test|significance test]] is a rule according to which values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507018.png" /> are considered significant if they exceed a prescribed critical value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507019.png" />. In its turn, the choice of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507020.png" /> is governed by the significance level, which equals the probability of the event <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507021.png" /> in the case that the hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507022.png" /> is true.
+
The probability of incorrectly rejecting the basic hypothesis being tested, when it is valid. In the theory of statistical hypotheses testing (cf. [[Statistical hypotheses, verification of|Statistical hypotheses, verification of]]), the significance level is also called the probability of an error of the first kind. The concept first arose in connection with the problem of testing for compatibility of a theory with experimental data. For example, suppose that observations are being conducted on the values of $n$ random variables $X_1,\dots,X_n$ and that, on the basis of these data, it is required to test a hypothesis $H$, according to which the joint distribution of $X_1,\dots,X_n$ has some specific property. An appropriate statistical test is constructed with the aid of a suitably selected function $Y=f(X_1,\dots,X_n)$; this function usually assumes small values when $H$ is true, and large values when $H$ is false. In particular, if $X_1,\dots,X_n$ are the outcomes of independent measurements (with error) of some known constant $a$ and the hypothesis $H$ states that no systematic errors are involved, then a reasonable choice of $Y$ is $(2m-n)^2$, where $m$ is the number of measured values of $X_i$ that exceed the true value $a$. A large observed value of $Y$ may be considered a significant statistical refutation of the hypothetical agreement between the experimental outcome and the hypothesis $H$. The corresponding [[Significance test|significance test]] is a rule according to which values of $Y$ are considered significant if they exceed a prescribed critical value $y$. In its turn, the choice of $y$ is governed by the significance level, which equals the probability of the event $\{Y>y\}$ in the case that the hypothesis $H$ is true.
  
Selection of a significance level should also take into account the unavoidable errors incurred when any specific significance level is employed. For example, if the significance level is excessively high, the main error will stem from rejection of a true hypothesis; but if the significance level is low, the error will usually arise from accepting a false hypothesis. In practice, the most commonly adopted significance levels in statistical calculations range from <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507023.png" /> to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507024.png" />. Significance levels lower than <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s085/s085070/s08507025.png" /> are used, for example, in statistical detection of toxic medical preparates, and also in other special situations where the overriding purpose is to ensure against incorrect rejection of the hypothesis being tested. See also [[Confidence estimation|Confidence estimation]].
+
Selection of a significance level should also take into account the unavoidable errors incurred when any specific significance level is employed. For example, if the significance level is excessively high, the main error will stem from rejection of a true hypothesis; but if the significance level is low, the error will usually arise from accepting a false hypothesis. In practice, the most commonly adopted significance levels in statistical calculations range from $0.01$ to $0.1$. Significance levels lower than $0.01$ are used, for example, in statistical detection of toxic medical preparates, and also in other special situations where the overriding purpose is to ensure against incorrect rejection of the hypothesis being tested. See also [[Confidence estimation|Confidence estimation]].
  
 
====References====
 
====References====

Latest revision as of 15:45, 4 November 2014

of a statistical test

The probability of incorrectly rejecting the basic hypothesis being tested, when it is valid. In the theory of statistical hypotheses testing (cf. Statistical hypotheses, verification of), the significance level is also called the probability of an error of the first kind. The concept first arose in connection with the problem of testing for compatibility of a theory with experimental data. For example, suppose that observations are being conducted on the values of $n$ random variables $X_1,\dots,X_n$ and that, on the basis of these data, it is required to test a hypothesis $H$, according to which the joint distribution of $X_1,\dots,X_n$ has some specific property. An appropriate statistical test is constructed with the aid of a suitably selected function $Y=f(X_1,\dots,X_n)$; this function usually assumes small values when $H$ is true, and large values when $H$ is false. In particular, if $X_1,\dots,X_n$ are the outcomes of independent measurements (with error) of some known constant $a$ and the hypothesis $H$ states that no systematic errors are involved, then a reasonable choice of $Y$ is $(2m-n)^2$, where $m$ is the number of measured values of $X_i$ that exceed the true value $a$. A large observed value of $Y$ may be considered a significant statistical refutation of the hypothetical agreement between the experimental outcome and the hypothesis $H$. The corresponding significance test is a rule according to which values of $Y$ are considered significant if they exceed a prescribed critical value $y$. In its turn, the choice of $y$ is governed by the significance level, which equals the probability of the event $\{Y>y\}$ in the case that the hypothesis $H$ is true.

Selection of a significance level should also take into account the unavoidable errors incurred when any specific significance level is employed. For example, if the significance level is excessively high, the main error will stem from rejection of a true hypothesis; but if the significance level is low, the error will usually arise from accepting a false hypothesis. In practice, the most commonly adopted significance levels in statistical calculations range from $0.01$ to $0.1$. Significance levels lower than $0.01$ are used, for example, in statistical detection of toxic medical preparates, and also in other special situations where the overriding purpose is to ensure against incorrect rejection of the hypothesis being tested. See also Confidence estimation.

References

[1] H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)


Comments

References

[a1] E.L. Lehmann, "Testing statistical hypotheses" , Wiley (1969)
How to Cite This Entry:
Significance level. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Significance_level&oldid=34286
This article was adapted from an original article by L.N. Bol'shev (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article