Markov process, stationary

A Markov process which is a stationary stochastic process. There is a stationary Markov process associated with a homogeneous Markov transition function if and only if there is a stationary initial distribution $\mu ( A)$ corresponding to this function, that is, $\mu ( A)$ satisfies

$$\mu ( A) = \int\limits _ { X } P ( x , t , A ) \mu ( d x ) .$$

If the phase space $X$ is finite, then a stationary initial distribution always exists, independent of whether the process has discrete $( t= 0 , 1 ,\dots)$ or continuous time. For a process in discrete time and for a countable set $X$, a condition for existence of a stationary distribution has been found by A.N. Kolmogorov [1]: It is necessary and sufficient that there is class of communicating states $Y \subset X$ such that the mathematical expectation of the time for reaching $y _ {2} \in Y$ from $y _ {1} \in Y$ is finite for any $y _ {1} \in Y$. This criterion has been generalized to strong Markov processes with an arbitrary phase space $X$: For the existence of a stationary process it is sufficient that there is a compact set $K \subset X$ such that the expectation of the time of reaching $K$ from $x$ is finite for all $x \in X$. There is the following sufficient condition for the existence of a stationary Markov process in terms of Lyapunov stochastic functions (cf. Lyapunov stochastic function): If there is a function $V ( x) \leq 0$ for which $L V ( x) \leq - 1$ for $x \notin K$, then there is a stationary Markov process associated with the Markov transition function $P ( x , t , A )$. Here $L$ is the infinitesimal generator of the process.

When the stationary initial distribution $\mu$ is unique, the corresponding stationary process is ergodic. In this case the Cesàro mean of the transition probabilities converges weakly to $\mu$. Under certain additional conditions,

$$\lim\limits _ {t \rightarrow \infty } P ( x , t , A ) = \ \mu ( A) \ ( \textrm{ weakly } ) .$$

A stationary initial distribution satisfies the Fokker–Planck(–Kolmogorov) equation $L ^ {*} \mu = 0$, where $L ^ {*}$ is the adjoint operator to the infinitesimal operator of the process. For example, $L ^ {*}$ is the adjoint operator to the generating differential operator of the process for diffusion processes. In this case $\mu$ has a density $p$ with respect to the Lebesgue measure which satisfies $L ^ {*} p = 0$. In the one-dimensional case this equation can be solved by quadrature.

References

[1]	A.N. Kolmogorov, "Markov chains with a countable number of states" , Moscow (1937) (In Russian)
[2]	J.L. Doob, "Stochastic processes" , Wiley (1953)
[3]	A.B. Sevast'yanov, "An ergodic theorem for Markov processes and its application to telephone systems with refusals" Theor. Probab. Appl. , 2 (1957) pp. 104–112 Teor. Veroyatnost. i Primenen. , 2 : 1 (1957) pp. 106–116

References

[a1]	K.L. Chung, "Markov chains with stationary transition probabilities", Springer (1960)
[a2]	W. Feller, "An introduction to probability theory and its applications", 1–2, Wiley (1966)
[a3]	P. Lévy, "Processus stochastiques et mouvement Brownien", Gauthier-Villars (1965)
[a4]	E. Parzen, "Stochastic processes", Holden-Day (1962)
[a5]	Yu.A. Rozanov, "Stationary random processes", Holden-Day (1967) (Translated from Russian)