Semi-Markov process

A stochastic process with a finite or countable set of states , having stepwise trajectories with jumps at times and such that the values at its jump times form a Markov chain with transition probabilities

The distributions of the jump times are described in terms of the distribution functions as follows:

(and, moreover, they are independent of the states of the process at earlier moments of time). If

for all , then the semi-Markov process is a continuous-time Markov chain. If all the distributions degenerate to a point, the result is a discrete-time Markov chain.

Semi-Markov processes provide a model for many processes in queueing theory and reliability theory. Related to semi-Markov processes are Markov renewal processes (see Renewal theory), which describe the number of times the process is in state during the time .

In analytic terms, the investigation of semi-Markov processes and Markov renewal processes reduces to a system of integral equations — the renewal equations.

References

[1]	V.S. Korolyuk, A.F. Turbin, "Semi-Markov processes and their applications" , Kiev (1976) (In Russian)

References

[a1]	E. Cinlar, "Introduction to stochastic processes" , Prentice-Hall (1975) pp. Chapt. 10