View source for Transition probabilities

<!--
t0937801.png
$#A+1 = 56 n = 0
$#C+1 = 56 : ~/encyclopedia/old_files/data/T093/T.0903780 Transition probabilities
Automatically converted into TeX, above some diagnostics.
Please remove this comment and the {{TEX|auto}} line below,
if TeX found to be correct.
-->

{{TEX|auto}}
{{TEX|done}}

The probabilities of transition of a [[Markov chain|Markov chain]]  $  \xi ( t) $
from a state  $  i $
into a state  $  j $
in a time interval  $  [ s, t] $:

$$ 
p _ {ij} ( s, t)  =  {\mathsf P} \{ \xi ( t) = j \mid  \xi ( s) = i \} ,\  s< t.
$$

In view of the basic property of a Markov chain, for any states  $  i, j \in S $ (where  $  S $
is the set of all states of the chain) and any  $  s < t < u $,

$$ 
p _ {ij} ( s, u)  =  \sum _ {k \in S } p _ {ik} ( s, t) p _ {kj} ( t, u).
$$

One usually considers homogeneous Markov chains, for which the transition probabilities  $  p _ {ij} ( s, t) $
depend on the length of  $  [ s, t] $
but not on its position on the time axis:

$$ 
p _ {ij} ( s, t)  =  p _ {ij} ( t- s).
$$

For any states  $  i $
and  $  j $
of a homogeneous Markov chain with discrete time, the sequence  $  p _ {ij} ( n) $
has a Cesàro limit, i.e.

$$ 
\lim\limits _ {n \rightarrow \infty }  
\frac{1}{n}
 \sum _ { k= 1} ^ { n }  p _ {ij} ( k)  \geq   0.
$$

Subject to certain additional conditions (and also for chains with continuous time), the limit exists also in the usual sense. See [[Markov chain, ergodic|Markov chain, ergodic]]; [[Markov chain, class of positive states of a|Markov chain, class of positive states of a]].

The transition probabilities  $  p _ {ij} ( t) $
for a Markov chain with discrete time are determined by the values of  $  p _ {ij} ( 1) $, 
$  i, j \in S $; 
for any  $  t > 0 $, 
$  i \in S $,

$$ 
\sum _ {j \in S } p _ {ij} ( t)  =  1.
$$

In the case of Markov chains with continuous time it is usually assumed that the transition probabilities satisfy the following additional conditions: All the  $  p _ {ij} ( t) $
are measurable as functions of  $  t \in ( 0, \infty ) $,

$$ 
\lim\limits _ {t \downarrow 0 }  p _ {ij} ( t)  =  0 \ \ 
( i \neq j),\ \ 
\lim\limits _ {t \downarrow 0 }  p _ {ii} ( t)  =  1 ,\ \ 
i, j \in S.
$$

Under these assumptions the following transition rates exist:

$$ \tag{1 }
\lambda _ {ij}  =  \lim\limits _ {t \downarrow 0 }  
\frac{1}{t}
 ( p _ {ij} ( t) - p _ {ij} ( 0))
 \leq   \infty ,\ \ 
i, j \in S;
$$

if all the  $  \lambda _ {ij} $
are finite and if  $  \sum _ {j \in S }  \lambda _ {ij} = 0 $, 
$  i \in S $, 
then the  $  p _ {ij} ( t) $
satisfy the Kolmogorov–Chapman system of differential equations

$$ \tag{2 }
p _ {ij}  ^  \prime  ( t)  =  \sum _ {k \in S } \lambda _ {ik} p _ {kj} ( t),\ \ 
p _ {ij}  ^  \prime  ( t)  =  \sum _ {k \in S } \lambda _ {kj} p _ {ik} ( t)
$$

with the initial conditions  $  p _ {ii} ( 0) = 1 $, 
$  p _ {ij} ( 0) = 0 $, 
$  i \neq j $, 
$  i, j \in S $ (see also [[Kolmogorov equation|Kolmogorov equation]]; [[Kolmogorov–Chapman equation|Kolmogorov–Chapman equation]]).

If a Markov chain is specified by means of the transition rates (1), then the transition probabilities  $  p _ {ij} ( t) $
satisfy the conditions

$$ 
p _ {ij} ( t)  \geq   0,\ \ 
\sum _ {j \in S } p _ {ij} ( t)  \leq   1,\ \ 
i, j \in S,\ \ 
t > 0;
$$

chains for which  $  \sum _ {j \in S }  p _ {ij} ( t) < 1 $
for certain  $  i \in S $
and  $  t > 0 $
are called defective (in this case the solution to (2) is not unique); if  $  \sum _ {j \in S }  p _ {ij} ( t) = 1 $
for all  $  i \in S $
and  $  t > 0 $, 
the chain is called proper.

Example. The Markov chain  $  \xi ( t) $
with set of states  $  \{ 0, 1 ,\dots \} $
and transition densities

$$ 
\lambda _ {i,i+1}  =  - \lambda _ {ii}  =  \lambda _ {i}  >  0,\ \ 
\lambda _ {ij}  =  0 \ \ 
( i \neq j \neq i+ 1)
$$

(i.e., a pure birth process) is defective if and only if

$$ 
\sum _ { i= 0} ^  \infty   
\frac{1}{\lambda _ {i} }
  <  \infty .
$$

Let

$$ 
\tau _ {0n}  =  \inf  \{ {t > 0 } : {\xi ( t) = n  ( \xi ( 0) = 0) } \}
,
$$

$$ 
\tau  =  \lim\limits _ {n \rightarrow \infty }  \tau _ {0n} ;
$$

then

$$ 
{\mathsf E} \tau  =  \sum _ { i= 1} ^  \infty   
\frac{1}{\lambda _ {i} }

$$

and for  $  {\mathsf E} \tau < \infty $
one has  $  {\mathsf P} \{ \tau < \infty \} = 1 $, 
i.e. the path of  $  \xi ( t) $"
tends to infinity in a finite time with probability 1"  (see also [[Branching processes, regularity of|Branching processes, regularity of]]).

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  K.L. Chung,   "Markov chains with stationary probability densities" , Springer  (1967)</TD></TR></table>

====Comments====
For additional references see also [[Markov chain|Markov chain]]; [[Markov process|Markov process]].

In (1),  $  \lambda _ {ij} \geq  0 $
if  $  i \neq j $
and  $  \lambda _ {ii} \leq  0 $.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  M. Iosifescu,   "Finite Markov processes and their applications" , Wiley  (1980)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  D. Revuz,   "Markov chains" , North-Holland  (1984)</TD></TR></table>