Transition probabilities
The probabilities of transition of a Markov chain
from a state i
into a state j
in a time interval [ s, t] :
p _ {ij} ( s, t) = {\mathsf P} \{ \xi ( t) = j \mid \xi ( s) = i \} ,\ s< t.
In view of the basic property of a Markov chain, for any states i, j \in S (where S is the set of all states of the chain) and any s < t < u ,
p _ {ij} ( s, u) = \sum _ {k \in S } p _ {ik} ( s, t) p _ {kj} ( t, u).
One usually considers homogeneous Markov chains, for which the transition probabilities p _ {ij} ( s, t) depend on the length of [ s, t] but not on its position on the time axis:
p _ {ij} ( s, t) = p _ {ij} ( t- s).
For any states i and j of a homogeneous Markov chain with discrete time, the sequence p _ {ij} ( n) has a Cesàro limit, i.e.
\lim\limits _ {n \rightarrow \infty } \frac{1}{n} \sum _ { k= 1} ^ { n } p _ {ij} ( k) \geq 0.
Subject to certain additional conditions (and also for chains with continuous time), the limit exists also in the usual sense. See Markov chain, ergodic; Markov chain, class of positive states of a.
The transition probabilities p _ {ij} ( t) for a Markov chain with discrete time are determined by the values of p _ {ij} ( 1) , i, j \in S ; for any t > 0 , i \in S ,
\sum _ {j \in S } p _ {ij} ( t) = 1.
In the case of Markov chains with continuous time it is usually assumed that the transition probabilities satisfy the following additional conditions: All the p _ {ij} ( t) are measurable as functions of t \in ( 0, \infty ) ,
\lim\limits _ {t \downarrow 0 } p _ {ij} ( t) = 0 \ \ ( i \neq j),\ \ \lim\limits _ {t \downarrow 0 } p _ {ii} ( t) = 1 ,\ \ i, j \in S.
Under these assumptions the following transition rates exist:
\tag{1 } \lambda _ {ij} = \lim\limits _ {t \downarrow 0 } \frac{1}{t} ( p _ {ij} ( t) - p _ {ij} ( 0)) \leq \infty ,\ \ i, j \in S;
if all the \lambda _ {ij} are finite and if \sum _ {j \in S } \lambda _ {ij} = 0 , i \in S , then the p _ {ij} ( t) satisfy the Kolmogorov–Chapman system of differential equations
\tag{2 } p _ {ij} ^ \prime ( t) = \sum _ {k \in S } \lambda _ {ik} p _ {kj} ( t),\ \ p _ {ij} ^ \prime ( t) = \sum _ {k \in S } \lambda _ {kj} p _ {ik} ( t)
with the initial conditions p _ {ii} ( 0) = 1 , p _ {ij} ( 0) = 0 , i \neq j , i, j \in S (see also Kolmogorov equation; Kolmogorov–Chapman equation).
If a Markov chain is specified by means of the transition rates (1), then the transition probabilities p _ {ij} ( t) satisfy the conditions
p _ {ij} ( t) \geq 0,\ \ \sum _ {j \in S } p _ {ij} ( t) \leq 1,\ \ i, j \in S,\ \ t > 0;
chains for which \sum _ {j \in S } p _ {ij} ( t) < 1 for certain i \in S and t > 0 are called defective (in this case the solution to (2) is not unique); if \sum _ {j \in S } p _ {ij} ( t) = 1 for all i \in S and t > 0 , the chain is called proper.
Example. The Markov chain \xi ( t) with set of states \{ 0, 1 ,\dots \} and transition densities
\lambda _ {i,i+1} = - \lambda _ {ii} = \lambda _ {i} > 0,\ \ \lambda _ {ij} = 0 \ \ ( i \neq j \neq i+ 1)
(i.e., a pure birth process) is defective if and only if
\sum _ { i= 0} ^ \infty \frac{1}{\lambda _ {i} } < \infty .
Let
\tau _ {0n} = \inf \{ {t > 0 } : {\xi ( t) = n ( \xi ( 0) = 0) } \} ,
\tau = \lim\limits _ {n \rightarrow \infty } \tau _ {0n} ;
then
{\mathsf E} \tau = \sum _ { i= 1} ^ \infty \frac{1}{\lambda _ {i} }
and for {\mathsf E} \tau < \infty one has {\mathsf P} \{ \tau < \infty \} = 1 , i.e. the path of \xi ( t) " tends to infinity in a finite time with probability 1" (see also Branching processes, regularity of).
References
[1] | K.L. Chung, "Markov chains with stationary probability densities" , Springer (1967) |
Comments
For additional references see also Markov chain; Markov process.
In (1), \lambda _ {ij} \geq 0 if i \neq j and \lambda _ {ii} \leq 0 .
References
[a1] | M. Iosifescu, "Finite Markov processes and their applications" , Wiley (1980) |
[a2] | D. Revuz, "Markov chains" , North-Holland (1984) |
Transition probabilities. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Transition_probabilities&oldid=52160