View source for Semi-Markov process

A [[Stochastic process|stochastic process]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842201.png" /> with a finite or countable set of states <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842202.png" />, having stepwise trajectories with jumps at times <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842203.png" /> and such that the values <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842204.png" /> at its jump times form a [[Markov chain|Markov chain]] with transition probabilities

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842205.png" /></td> </tr></table>

The distributions of the jump times <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842206.png" /> are described in terms of the distribution functions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842207.png" /> as follows:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842208.png" /></td> </tr></table>

(and, moreover, they are independent of the states of the process at earlier moments of time). If

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s0842209.png" /></td> </tr></table>

for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s08422010.png" />, then the semi-Markov process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s08422011.png" /> is a continuous-time Markov chain. If all the distributions degenerate to a point, the result is a discrete-time Markov chain.

Semi-Markov processes provide a model for many processes in [[Queueing theory|queueing theory]] and [[Reliability theory|reliability theory]]. Related to semi-Markov processes are Markov renewal processes (see [[Renewal theory|Renewal theory]]), which describe the number of times the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s08422012.png" /> is in state <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s08422013.png" /> during the time <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s084/s084220/s08422014.png" />.

In analytic terms, the investigation of semi-Markov processes and Markov renewal processes reduces to a system of integral equations — the renewal equations.

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  V.S. Korolyuk,   A.F. Turbin,   "Semi-Markov processes and their applications" , Kiev  (1976)  (In Russian)</TD></TR></table>



====Comments====


====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  E. Cinlar,   "Introduction to stochastic processes" , Prentice-Hall  (1975)  pp. Chapt. 10</TD></TR></table>