Markov process, stationary

A Markov process which is a stationary stochastic process. There is a stationary Markov process associated with a homogeneous Markov transition function if and only if there is a stationary initial distribution corresponding to this function, that is, satisfies

If the phase space is finite, then a stationary initial distribution always exists, independent of whether the process has discrete or continuous time. For a process in discrete time and for a countable set , a condition for existence of a stationary distribution has been found by A.N. Kolmogorov [1]: It is necessary and sufficient that there is class of communicating states such that the mathematical expectation of the time for reaching from is finite for any . This criterion has been generalized to strong Markov processes with an arbitrary phase space : For the existence of a stationary process it is sufficient that there is a compact set such that the expectation of the time of reaching from is finite for all . There is the following sufficient condition for the existence of a stationary Markov process in terms of Lyapunov stochastic functions (cf. Lyapunov stochastic function): If there is a function for which for , then there is a stationary Markov process associated with the Markov transition function . Here is the infinitesimal generator of the process.

When the stationary initial distribution is unique, the corresponding stationary process is ergodic. In this case the Cesàro mean of the transition probabilities converges weakly to . Under certain additional conditions,

A stationary initial distribution satisfies the Fokker–Planck(–Kolmogorov) equation , where is the adjoint operator to the infinitesimal operator of the process. For example, is the adjoint operator to the generating differential operator of the process for diffusion processes. In this case has a density with respect to the Lebesgue measure which satisfies . In the one-dimensional case this equation can be solved by quadrature.

References

[1]	A.N. Kolmogorov, "Markov chains with a countable number of states" , Moscow (1937) (In Russian)
[2]	J.L. Doob, "Stochastic processes" , Wiley (1953)
[3]	A.B. Sevast'yanov, "An ergodic theorem for Markov processes and its application to telephone systems with refusals" Theor. Probab. Appl. , 2 (1957) pp. 104–112 Teor. Veroyatnost. i Primenen. , 2 : 1 (1957) pp. 106–116

References

[a1]	K.L. Chung, "Markov chains with stationary transition probabilities", Springer (1960)
[a2]	W. Feller, "An introduction to probability theory and its applications", 1–2, Wiley (1966)
[a3]	P. Lévy, "Processus stochastiques et mouvement Brownien", Gauthier-Villars (1965)
[a4]	E. Parzen, "Stochastic processes", Holden-Day (1962)
[a5]	Yu.A. Rozanov, "Stationary random processes", Holden-Day (1967) (Translated from Russian)