Markov process, stationary
A Markov process which is a stationary stochastic process. There is a stationary Markov process associated with a homogeneous Markov transition function if and only if there is a stationary initial distribution corresponding to this function, that is,
satisfies
![]() |
If the phase space is finite, then a stationary initial distribution always exists, independent of whether the process has discrete
or continuous time. For a process in discrete time and for a countable set
, a condition for existence of a stationary distribution has been found by A.N. Kolmogorov [1]: It is necessary and sufficient that there is class of communicating states
such that the mathematical expectation of the time for reaching
from
is finite for any
. This criterion has been generalized to strong Markov processes with an arbitrary phase space
: For the existence of a stationary process it is sufficient that there is a compact set
such that the expectation of the time of reaching
from
is finite for all
. There is the following sufficient condition for the existence of a stationary Markov process in terms of Lyapunov stochastic functions (cf. Lyapunov stochastic function): If there is a function
for which
for
, then there is a stationary Markov process associated with the Markov transition function
. Here
is the infinitesimal generator of the process.
When the stationary initial distribution is unique, the corresponding stationary process is ergodic. In this case the Cesàro mean of the transition probabilities converges weakly to
. Under certain additional conditions,
![]() |
A stationary initial distribution satisfies the Fokker–Planck(–Kolmogorov) equation , where
is the adjoint operator to the infinitesimal operator of the process. For example,
is the adjoint operator to the generating differential operator of the process for diffusion processes. In this case
has a density
with respect to the Lebesgue measure which satisfies
. In the one-dimensional case this equation can be solved by quadrature.
References
[1] | A.N. Kolmogorov, "Markov chains with a countable number of states" , Moscow (1937) (In Russian) |
[2] | J.L. Doob, "Stochastic processes" , Wiley (1953) |
[3] | A.B. Sevast'yanov, "An ergodic theorem for Markov processes and its application to telephone systems with refusals" Theor. Probab. Appl. , 2 (1957) pp. 104–112 Teor. Veroyatnost. i Primenen. , 2 : 1 (1957) pp. 106–116 |
Comments
References
[a1] | K.L. Chung, "Markov chains with stationary transition probabilities" , Springer (1960) |
[a2] | W. Feller, "An introduction to probability theory and its applications" , 1–2 , Wiley (1966) |
[a3] | P. Lévy, "Processus stochastiques et mouvement Brownien" , Gauthier-Villars (1965) |
[a4] | E. Parzen, "Stochastic processes" , Holden-Day (1962) |
[a5] | Yu.A. Rozanov, "Stationary random processes" , Holden-Day (1967) (Translated from Russian) |
Markov process, stationary. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Markov_process,_stationary&oldid=25956