Namespaces
Variants
Actions

Semi-Markov process

From Encyclopedia of Mathematics
Jump to: navigation, search


A stochastic process with a finite or countable set of states N = \{ 1 , 2 , . . . \} , having stepwise trajectories with jumps at times 0 < \tau _ {1} < \tau _ {2} < \dots and such that the values X ( \tau _ {n} ) at its jump times form a Markov chain with transition probabilities

p _ {ij} = {\mathsf P} \{ X ( \tau _ {n} ) = j \mid X ( \tau _ {n-} 1 ) = i \} .

The distributions of the jump times \tau _ {n} are described in terms of the distribution functions F _ {ij} ( x) as follows:

{\mathsf P} \{ \tau _ {n} - \tau _ {n-} 1 \leq x ,\ X ( \tau _ {n} ) = j \mid X ( \tau _ {n-} 1 ) = i \} = p _ {ij} F _ {ij} ( x)

(and, moreover, they are independent of the states of the process at earlier moments of time). If

F _ {ij} ^ { \prime } ( x) = e ^ {- \lambda _ {ij} x } ,\ \ x \geq 0 ,

for all i , j \in N , then the semi-Markov process X ( t) is a continuous-time Markov chain. If all the distributions degenerate to a point, the result is a discrete-time Markov chain.

Semi-Markov processes provide a model for many processes in queueing theory and reliability theory. Related to semi-Markov processes are Markov renewal processes (see Renewal theory), which describe the number of times the process X ( t) is in state i \in N during the time [ 0 , t ] .

In analytic terms, the investigation of semi-Markov processes and Markov renewal processes reduces to a system of integral equations — the renewal equations.

References

[1] V.S. Korolyuk, A.F. Turbin, "Semi-Markov processes and their applications" , Kiev (1976) (In Russian)

Comments

References

[a1] E. Cinlar, "Introduction to stochastic processes" , Prentice-Hall (1975) pp. Chapt. 10
How to Cite This Entry:
Semi-Markov process. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Semi-Markov_process&oldid=48652
This article was adapted from an original article by B.A. Sevast'yanov (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article