View source for Bernoulli measure

A measure describing the independent repetition of an experiment with two possible outcomes, such as playing head and tail with a possibly biased coin. This is the simplest and most basic probabilistic scheme for which the fundamental theorems of [[Probability theory|probability theory]], such as the weak and strong law of large numbers, the integral and local limit properties and the large deviation principle, were first and most easily proved.

The scheme can be generalized to experiments with possibly more than two outcomes and can be mathematically formalized in the following way. Given a countable set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103301.png" /> and a finite set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103302.png" /> with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103303.png" /> elements <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103304.png" />, which can be identified with the set of the first <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103305.png" /> positive integers if there are no special requirements, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103306.png" /> non-negative numbers <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103307.png" /> with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103308.png" />, one defines the corresponding Bernoulli measure on the space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b1103309.png" /> as a [[Probability measure|probability measure]] (finitely or countably additive, in accordance with the setting) <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033010.png" /> such that the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033011.png" />'s are independent and identically distributed random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033012.png" /> for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033013.png" />. In the countable additive framework, a basic result is the Kolmogorov [[Zero-one law|zero-one law]] [[#References|[a9]]], stating that events in the algebra at infinity (i.e., events that are measurable with respect to the Borel fields generated by the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033014.png" />'s in the complement of every finite subset of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033015.png" />) have probability either <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033016.png" /> or <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033017.png" />.

Percolation theory on Bernoulli measures is a theory that, although in a simpler setting, presents in many respects the same kinds of phenomena as statistical mechanics, with important difficult results and many open problems (see [[Statistical mechanics, mathematical problems in|Statistical mechanics, mathematical problems in]]; [[#References|[a8]]], [[#References|[a3]]]). In the most common setting, one considers on each site <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033018.png" /> of a lattice a [[Random variable|random variable]] with values <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033019.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033020.png" />; such variables are assumed to be independent and identically distributed, so that one finds that the Bernoulli measures parametrized by the probability <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033021.png" /> for an <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033022.png" /> are equal to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033023.png" />. Many events considered in percolation theory are increasing (or positive) in the sense that their indicator functions are non-decreasing functions of the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033024.png" />'s. L. Russo [[#References|[a7]]] has proved a finite version of Kolmogorov's zero-one law; it states that increasing events that in a suitable sense depend little on each <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033025.png" /> have probability close to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033026.png" /> or <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033027.png" /> for all but a small interval of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033028.png" />'s. T.E. Harris [[#References|[a4]]] has proved a basic inequality for increasing events on Bernoulli measures: if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033029.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033030.png" /> are increasing events, then <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033031.png" />. An inequality in the opposite sense was proved by J. van den Berg and H. Kesten [[#References|[a1]]] for increasing events; it was generalized to arbitrary events by D. Reimer [[#References|[a6]]]. It states that the probability that two events happen disjointly (in the sense that one can verify their occurrence by looking at disjoint subsets of the lattice) is less than or equal to the product of their probabilities.

The [[De Finetti theorem|De Finetti theorem]] on infinite sequences of exchangeable random events [[#References|[a2]]] shows that Bernoulli measures are relevant for statistical inference in a wide range of situations. It states that if the distribution of an infinite family of random events is invariant under finite permutations, then it can be expressed as a mixture of Bernoulli measures. The assumption of exchangeability is very natural in many concrete situations.

Let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033032.png" /> be the set of integer numbers and let <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033033.png" /> be the left shift operator on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033034.png" />:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033035.png" /></td> </tr></table>

The pair consisting of the Bernoulli measure and the shift operator is called the Bernoulli shift. One says that an invertible measurable mapping <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033036.png" /> between two Bernoulli shifts, with probability measures <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033037.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033038.png" /> and shift operators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033039.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033040.png" />, is an isomorphism between Bernoulli shifts if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033041.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033042.png" />. The famous Ornstein theorem [[#References|[a5]]], which has been generalized in many ways, states that two Bernoulli shifts are isomorphic if and only if their Kolmogorov–Sinai entropies are equal. For a Bernoulli shift the Kolmogorov–Sinai entropy is given by <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033043.png" />, with the convention that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033044.png" /> for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/b/b110/b110330/b11033045.png" />.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  J. van den Berg,   H. Kesten,   "Inequalities with applications to percolation and reliability theory"  ''J. Appl. Probab.'' , '''22'''  (1985)  pp. 556</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  B. de Finetti,   "La prévision: ses lois logiques, ses sources subjectives"  ''Ann. Inst. Poincaré'' , '''VII'''  (1937)  pp. 1–68</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  G. Grimmett,   "Percolation" , Springer  (1989)</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  T.E. Harris,   "A lower bound for the critical probability in certain percolation processes"  ''Proc. Cambridge Philos. Soc.'' , '''56'''  (1960)  pp. 13</TD></TR><TR><TD valign="top">[a5]</TD> <TD valign="top">  D. Ornstein,   "Ergodic theory, randomness, and dynamical systems." , Yale Univ. Press  (1974)</TD></TR><TR><TD valign="top">[a6]</TD> <TD valign="top">  D. Reimer,   "Butterflies"  ''Preprint''  (1994)</TD></TR><TR><TD valign="top">[a7]</TD> <TD valign="top">  L. Russo,   "An approximate zero-one law"  ''Z. Wahrscheinlichkeitsth. verw. Gebiete'' , '''61'''  (1982)  pp. 129–139</TD></TR><TR><TD valign="top">[a8]</TD> <TD valign="top">  H. Kesten,   "Percolation theory for mathematicians" , ''Progress in Probab. and Stat.'' , '''2''' , Birkhäuser  (1983)</TD></TR><TR><TD valign="top">[a9]</TD> <TD valign="top">  A.N. Kolmogorov,   "Foundations of the theory of probability" , Chelsea  (1950)  (In Russian)</TD></TR></table>