View source for Law of large numbers

{{MSC|60F05}}

[[Category:Limit theorems]]

A general principle according to which under certain very general conditions the simultaneous action of random factors leads to a result which is practically non-random. That the frequency of occurrence of a random event tends to become equal to its probability as the number of trials increases (a phenomenon which was probably first noted for games of chance) may serve as the first example of this principle.

At the turn of the 17th century J. Bernoulli [[#References|[1]]], [[#References|[12]]] demonstrated a theorem stating that, in a sequence of independent trials, in each of which the probability of occurrence of a certain event <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577201.png" /> has the same value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577202.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577203.png" />, the relationship

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577204.png" /></td> <td valign="top" style="width:5%;text-align:right;">(1)</td></tr></table>

is valid for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577205.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577206.png" />; here, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577207.png" /> is the number of occurrences of the event in the first <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577208.png" /> trials and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l0577209.png" /> is the frequency of the occurrence. This [[Bernoulli theorem|Bernoulli theorem]] was extended by S. Poisson [[#References|[2]]] to the case of a sequence of independent trials in which the probability of the occurrence of an event <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772010.png" /> varies with the number of the trial. Let this probability in the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772011.png" />-th trial be <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772012.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772013.png" /> and let

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772014.png" /></td> </tr></table>

Then, according to the [[Poisson theorem|Poisson theorem]],

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772015.png" /></td> <td valign="top" style="width:5%;text-align:right;">(2)</td></tr></table>

for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772016.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772017.png" />. A rigorous proof of this theorem was first given in 1846 by P.L. Chebyshev, whose method was quite different from that of Poisson and was based on certain extremal considerations, whereas Poisson deduced (2) from an approximate formula for the above probability, based on a law of Gauss which at that time had not yet been proved rigorously. Poisson was the first to use the term "law of large numbers" , by which he denoted his own generalization of the Bernoulli theorem.

A further natural extension of the Bernoulli and Poisson theorems is a consequence of the fact that the random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772018.png" /> may be represented as the sum

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772019.png" /></td> </tr></table>

of independent random variables, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772020.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772021.png" /> occurs in the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772022.png" />-th trial and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772023.png" /> otherwise. The mathematical expectation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772024.png" /> (which is identical with the average of the mathematical expectations <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772025.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772026.png" />) is <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772027.png" /> in the case of Bernoulli and is <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772028.png" /> in the case of Poisson. In other words, in both cases one deals with a deviation of the arithmetical average of the variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772029.png" /> from the arithmetical average of their mathematical expectations.

In his work "On average quantities" , which appeared in 1867, Chebyshev established that the relationship

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772030.png" /></td> <td valign="top" style="width:5%;text-align:right;">(3)</td></tr></table>

is valid, for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772031.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772032.png" />, for independent random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772033.png" /> under fairly general assumptions. Chebyshev assumed that the mathematical expectations <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772034.png" /> are all bounded by the same constant, even though his proof makes it clear that the requirement of boundedness of the variances of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772035.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772036.png" />, or even the requirement

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772037.png" /></td> </tr></table>

is sufficient. It was thus shown by Chebyshev that the Bernoulli theorem is susceptible of extensive generalizations. A.A. Markov noted the possibility of further extensions and proposed to apply the term "law of large numbers" to all extensions of the Bernoulli theorem (and, in particular, to (3)). Chebyshev's method is based on a rigorous formulation of all the properties of the mathematical expectations and on the use of the so- called [[Chebyshev inequality in probability theory|Chebyshev inequality in probability theory]]. For the probability (3) it yields an estimate of the form

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772038.png" /></td> </tr></table>

This estimate may be replaced by a more exact one — but under more restrictive conditions, see [[Bernstein inequality|Bernstein inequality]]. Subsequent proofs of the law of large numbers are all, to varying extents, developments of Chebyshev's method. Using an appropriate "truncation" of the random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772039.png" /> (replacement of these variables by auxiliary variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772040.png" />, viz. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772041.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772042.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772043.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772044.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772045.png" /> is a constant), Markov extended the law of large numbers to the case when the variance of the terms does not exist. He showed, for instance, that (3) is valid if, for certain constants <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772046.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772047.png" /> and for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772048.png" />,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772049.png" /></td> </tr></table>

Khinchin's theorem (1929) is proved in a similar manner: If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772050.png" /> have the same distribution and if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772051.png" /> exists, then the law of large numbers (3) is valid.

It is possible to formulate more or less final versions of the law of large numbers for sums of independent random variables. To do this, it is expedient to adopt a more general point of view, involving the concept of a sequence of asymptotically-constant random variables. The random variables of a sequence <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772052.png" /> are said to be asymptotically constant if there exists a sequence of constants <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772053.png" /> such that for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772054.png" />, if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772055.png" />,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772056.png" /></td> <td valign="top" style="width:5%;text-align:right;">(4)</td></tr></table>

(i.e. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772057.png" /> converges to zero in probability). If (4) is valid for some <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772058.png" />, it is also valid for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772059.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772060.png" /> is the median (cf. [[Median (in statistics)|Median (in statistics)]]) of the random variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772061.png" />. Furthermore, instead of the sequence <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772062.png" /> of independent random variables, one may also take a so-called [[Triangular array|triangular array]]:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772063.png" /></td> </tr></table>

of random variables (the first subscript is the row number, while the second indicates the number of the variable within the row). The random variables of each individual row are assumed to be mutually independent. A sequence of partial sums is readily reduced to a triangular array if one assumes that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772064.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772065.png" /> <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772066.png" />.

Let

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772067.png" /></td> </tr></table>

The general problem of the applicability of the law of large numbers to sums of independent random variables may then be stated as follows: Under what conditions are the sums <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772068.png" /> asymptotically constant?

This question was answered in 1928 by A.N. Kolmogorov. Assume, without loss of generality, that the medians of the variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772069.png" /> are zero. Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772070.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772071.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772072.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772073.png" />. The simultaneous fulfillment of the two conditions

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772074.png" /></td> </tr></table>

and

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772075.png" /></td> </tr></table>

is necessary and sufficient for the sums <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772076.png" /> to be asymptotically constant. The sum <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772077.png" /> may be taken as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772078.png" />. That these conditions are sufficient is readily demonstrated by Chebyshev's method. If the mathematical expectations <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772079.png" /> exist, it is easy to find supplementary conditions under which it is permissible to choose <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772080.png" />, which yields necessary and sufficient conditions for the law of large numbers in the classical formulation (3). For a sequence of independent identically-distributed variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772081.png" /> these conditions are reduced — in accordance with Khinchin's theorem quoted above — to the existence of the mathematical expectation. At the same time, the condition

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772082.png" /></td> <td valign="top" style="width:5%;text-align:right;">(5)</td></tr></table>

is necessary and sufficient for the arithmetical averages <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772083.png" /> to be asymptotically constant. Examples of cases in which condition (5) is not met are easily found. Thus, the condition is not met if all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772084.png" /> have a [[Cauchy distribution|Cauchy distribution]] with density <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772085.png" /> (to which corresponds the characteristic function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772086.png" />). Here, the arithmetical averages <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772087.png" /> have the characteristic function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772088.png" /> and, as a result, have the same distribution for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772089.png" /> as do the individual terms.

The most important cases to which the law of large numbers does not apply involve the times of return to the starting point in a random walk. Thus, in a symmetric [[Bernoulli random walk|Bernoulli random walk]] the time <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772090.png" /> passing prior to the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772091.png" />-th return to the starting point is the sum of the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772092.png" /> independent random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772093.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772094.png" /> is the time passed prior to the first return, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772095.png" /> is the time between the first and the second return, etc. The distribution of the variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772096.png" /> converges as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772097.png" /> to a non-degenerate limit distribution with density

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772098.png" /></td> </tr></table>

and zero for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l05772099.png" />. Accordingly, in this case the distribution of the arithmetical average of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720100.png" />, i.e. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720101.png" />, is located, roughly speaking, on a segment of length of order <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720102.png" /> (whereas, in cases where the law of large numbers is applicable, it is located on segments of length <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720103.png" />).

The applicability of the law of large numbers to sums of dependent variables (both in its classical and in its more general formulations) is connected, first and foremost, with the asymptotic independence of the random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720104.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720105.png" /> as the difference between their subscripts <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720106.png" /> increases. The corresponding theorems were first proved by Markov in 1907 for variables connected in a [[Markov chain|Markov chain]]. In fact, let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720107.png" /> be a homogeneous Markov chain with finite number of states, and let all one-step transition probabilities be positive. Here, the asymptotic independence of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720108.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720109.png" /> as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720110.png" /> follows from the fact that the conditional distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720111.png" /> under a fixed value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720112.png" /> tends, as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720113.png" />, to a limit which does not depend on the chosen value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720114.png" /> (Markov's ergodic theorem). The law of large numbers is deduced from this theorem: It is first established that, as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720115.png" />,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720116.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720117.png" />; hence it follows that as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720118.png" />,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720119.png" /></td> </tr></table>

The following, more general case, is governed by Bernstein's conditions: If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720120.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720121.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720122.png" /> is a constant, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720123.png" /> is the correlation coefficient and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720124.png" /> is a function which tends to zero as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720125.png" />, then the law of large numbers (3) is applicable to the variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720126.png" />. For sequences <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720127.png" /> which are stationary in the wide sense, the correlation condition may be weakened somewhat, by replacing it by

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720128.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720129.png" />.

The above-mentioned results may be extended in various ways. First, only convergence in probability was considered so far. Other types of convergence may also be considered, including convergence with probability one, mean-square convergence, etc. (in fact, many of the above conditions ensure mean-square convergence, which implies convergence in probability). The case of convergence with probability one is important, and for this reason has been especially named the [[Strong law of large numbers|strong law of large numbers]].

Furthermore, many of these theorems can be applied, with suitable modifications, to random vectors having values in Euclidean spaces of any dimension, in a Hilbert space or in certain Banach spaces. For instance, if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720130.png" /> is a sequence of independent identically-distributed random vectors with values in a separable Banach space, and if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720131.png" /> (where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720132.png" /> is the norm of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720133.png" />) is finite, one has

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720134.png" /></td> </tr></table>

for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720135.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720136.png" />.

The law of large numbers, when considered in its most general form, is closely related to ergodic theorems (cf. [[Ergodic theorem|Ergodic theorem]]). Clearly, many theorems are also applicable to the case of the average <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720137.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720138.png" /> is a random process depending on a continuous parameter (see, for example, [[#References|[10]]]).

Finally, instead of the sums of random variables one may consider other symmetric functions of them. This was in fact done by A.Ya. Khinchin (1951–1955) to prove certain conclusions in statistical mechanics [[#References|[9]]]. The result obtained by him may be illustrated by the following example. Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720139.png" /> be the coordinates of a point uniformly distributed on the surface of the sphere

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720140.png" /></td> </tr></table>

The law of large numbers then applies to a wide class of symmetric functions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720141.png" /> in the sense that as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l057/l057720/l057720142.png" />, their values are asymptotically constant (this is similar to the observation made in 1925 by P. Lévy to the effect that sufficiently regular functions of a very large number of variables are almost constant in a large part of their domain of definition).

Extensive statistical data, illustrating the law of large numbers, can be found in older textbooks; see, for example, [[#References|[4]]], [[#References|[11]]].

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top"> J. Bernoulli, "Ars conjectandi" , ''Werke'' , '''3''' , Birkhäuser (1975) pp. 107–286 (Original: Basle, 1713) {{MR|2349550}} {{MR|2393219}} {{MR|0935946}} {{MR|0850992}} {{MR|0827905}} {{ZBL|0957.01032}} {{ZBL|0694.01020}} {{ZBL|30.0210.01}} </TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top"> S.D. Poisson, "Récherches sur la probabilité des jugements en matière criminelle et en matière civile" , Paris (1837)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top"> P.L. Chebyshev, "Oeuvres de P.L. Tchebycheff" , '''2''' , Chelsea (1961) (Translated from Russian)</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top"> A.A. Markov, "Wahrscheinlichkeitsrechung" , Teubner (1912) (Translated from Russian) {{MR|}} {{ZBL|39.0292.02}} </TD></TR><TR><TD valign="top">[5]</TD> <TD valign="top"> S.N. Bernshtein, "Probability theory" , Moscow-Leningrad (1946) (In Russian) {{MR|1868030}} {{ZBL|}} </TD></TR><TR><TD valign="top">[6]</TD> <TD valign="top"> B.V. Gnedenko, A.N. Kolmogorov, "Limit distributions for sums of independent random variables" , Addison-Wesley (1954) (Translated from Russian) {{MR|0062975}} {{ZBL|0056.36001}} </TD></TR><TR><TD valign="top">[7]</TD> <TD valign="top"> J.L. Doob, "Stochastic processes" , Chapman &amp; Hall (1953) {{MR|1570654}} {{MR|0058896}} {{ZBL|0053.26802}} </TD></TR><TR><TD valign="top">[8]</TD> <TD valign="top"> U. Grenander, "Probabilities on algebraic structures" , Wiley (1963) {{MR|0206994}} {{ZBL|0131.34804}} </TD></TR><TR><TD valign="top">[9]</TD> <TD valign="top"> A.Ya. Khinchin, "Symmetric functions on multi-dimensional surfaces" , ''To the memory of A.A. Adronov'' , Moscow (1955) pp. 541–574 (In Russian) {{MR|74335}} {{ZBL|}} </TD></TR><TR><TD valign="top">[10]</TD> <TD valign="top"> M. Loève, "Probability theory" , Princeton Univ. Press (1963) {{MR|0203748}} {{ZBL|0108.14202}} </TD></TR><TR><TD valign="top">[11]</TD> <TD valign="top"> J.V. Uspensky, "Introduction to mathematical probability" , McGraw-Hill (1937) {{MR|1524355}} {{ZBL|63.1069.01}} </TD></TR><TR><TD valign="top">[12]</TD> <TD valign="top"> J. Bernoulli, "On the law of large numbers" , Moscow (1986) (In Russian) {{MR|}} {{ZBL|0646.01008}} </TD></TR></table>



====Comments====


====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top"> P. Révész, "The laws of large numbers" , Acad. Press (1968) {{MR|0245079}} {{ZBL|0203.50403}} </TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top"> J. Hoffman-Jørgensen, G. Pisier, "The law of large numbers and the central limit theorem in Banach spaces" ''Ann. Prob.'' , '''4''' (1976) pp. 587–599 {{MR|423451}} {{ZBL|}} </TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top"> P. Hall, C.C. Heyde, "Martingale limit theory and its application" , Acad. Press (1980) {{MR|0624435}} {{ZBL|0462.60045}} </TD></TR></table>