View source for Transition probabilities

The probabilities of transition of a [[Markov chain|Markov chain]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937801.png" /> from a state <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937802.png" /> into a state <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937803.png" /> in a time interval <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937804.png" />:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937805.png" /></td> </tr></table>

In view of the basic property of a Markov chain, for any states <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937806.png" /> (where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937807.png" /> is the set of all states of the chain) and any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937808.png" />,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t0937809.png" /></td> </tr></table>

One usually considers homogeneous Markov chains, for which the transition probabilities <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378010.png" /> depend on the length of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378011.png" /> but not on its position on the time axis:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378012.png" /></td> </tr></table>

For any states <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378013.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378014.png" /> of a homogeneous Markov chain with discrete time, the sequence <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378015.png" /> has a Cesàro limit, i.e.

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378016.png" /></td> </tr></table>

Subject to certain additional conditions (and also for chains with continuous time), the limit exists also in the usual sense. See [[Markov chain, ergodic|Markov chain, ergodic]]; [[Markov chain, class of positive states of a|Markov chain, class of positive states of a]].

The transition probabilities <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378017.png" /> for a Markov chain with discrete time are determined by the values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378018.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378019.png" />; for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378020.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378021.png" />,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378022.png" /></td> </tr></table>

In the case of Markov chains with continuous time it is usually assumed that the transition probabilities satisfy the following additional conditions: All the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378023.png" /> are measurable as functions of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378024.png" />,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378025.png" /></td> </tr></table>

Under these assumptions the following transition rates exist:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378026.png" /></td> <td valign="top" style="width:5%;text-align:right;">(1)</td></tr></table>

if all the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378027.png" /> are finite and if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378028.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378029.png" />, then the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378030.png" /> satisfy the Kolmogorov–Chapman system of differential equations

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378031.png" /></td> <td valign="top" style="width:5%;text-align:right;">(2)</td></tr></table>

with the initial conditions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378032.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378033.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378034.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378035.png" /> (see also [[Kolmogorov equation|Kolmogorov equation]]; [[Kolmogorov–Chapman equation|Kolmogorov–Chapman equation]]).

If a Markov chain is specified by means of the transition rates (1), then the transition probabilities <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378036.png" /> satisfy the conditions

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378037.png" /></td> </tr></table>

chains for which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378038.png" /> for certain <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378039.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378040.png" /> are called defective (in this case the solution to (2) is not unique); if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378041.png" /> for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378042.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378043.png" />, the chain is called proper.

Example. The Markov chain <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378044.png" /> with set of states <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378045.png" /> and transition densities

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378046.png" /></td> </tr></table>

(i.e., a pure birth process) is defective if and only if

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378047.png" /></td> </tr></table>

Let

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378048.png" /></td> </tr></table>

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378049.png" /></td> </tr></table>

then

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378050.png" /></td> </tr></table>

and for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378051.png" /> one has <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378052.png" />, i.e. the path of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378053.png" />  "tends to infinity in a finite time with probability 1"  (see also [[Branching processes, regularity of|Branching processes, regularity of]]).

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  K.L. Chung,   "Markov chains with stationary probability densities" , Springer  (1967)</TD></TR></table>



====Comments====
For additional references see also [[Markov chain|Markov chain]]; [[Markov process|Markov process]].

In (1), <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378054.png" /> if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378055.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/t/t093/t093780/t09378056.png" />.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  M. Iosifescu,   "Finite Markov processes and their applications" , Wiley  (1980)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  D. Revuz,   "Markov chains" , North-Holland  (1984)</TD></TR></table>