Transition probabilities

The probabilities of transition of a Markov chain from a state into a state in a time interval :

In view of the basic property of a Markov chain, for any states (where is the set of all states of the chain) and any ,

One usually considers homogeneous Markov chains, for which the transition probabilities depend on the length of but not on its position on the time axis:

For any states and of a homogeneous Markov chain with discrete time, the sequence has a Cesàro limit, i.e.

Subject to certain additional conditions (and also for chains with continuous time), the limit exists also in the usual sense. See Markov chain, ergodic; Markov chain, class of positive states of a.

The transition probabilities for a Markov chain with discrete time are determined by the values of , ; for any , ,

In the case of Markov chains with continuous time it is usually assumed that the transition probabilities satisfy the following additional conditions: All the are measurable as functions of ,

Under these assumptions the following transition rates exist:

(1)

if all the are finite and if , , then the satisfy the Kolmogorov–Chapman system of differential equations

(2)

with the initial conditions , , , (see also Kolmogorov equation; Kolmogorov–Chapman equation).

If a Markov chain is specified by means of the transition rates (1), then the transition probabilities satisfy the conditions

chains for which for certain and are called defective (in this case the solution to (2) is not unique); if for all and , the chain is called proper.

Example. The Markov chain with set of states and transition densities

(i.e., a pure birth process) is defective if and only if

Let

then

and for one has , i.e. the path of "tends to infinity in a finite time with probability 1" (see also Branching processes, regularity of).

References

[1]	K.L. Chung, "Markov chains with stationary probability densities" , Springer (1967)

Comments

For additional references see also Markov chain; Markov process.

In (1), if and .

References

[a1]	M. Iosifescu, "Finite Markov processes and their applications" , Wiley (1980)
[a2]	D. Revuz, "Markov chains" , North-Holland (1984)