View source for Lyapunov stochastic function

A non-negative function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611801.png" /> for which the pair <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611802.png" /> is a supermartingale for some random process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611803.png" /> up to the instant <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611804.png" /> (cf. also [[Martingale|Martingale]]). If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611805.png" /> is a [[Markov process|Markov process]], then the Lyapunov stochastic function is a function for which the Lyapunov stochastic operator

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611806.png" /></td> </tr></table>

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611807.png" /></td> </tr></table>

is non-positive. The operator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611808.png" /> is the [[Infinitesimal operator|infinitesimal operator]] of the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l0611809.png" />, and so the verification of the condition <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118010.png" /> is easily carried out in specific cases. The operator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118011.png" /> goes into the usual Lyapunov operator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118012.png" /> when the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118013.png" /> is determinate and is described by a system of differential equations. By means of the Lyapunov stochastic function it is possible to verify a number of qualitative properties of the trajectories of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118014.png" />; their role in the theory of random processes is similar to the role of the classical [[Lyapunov function|Lyapunov function]] in the theory of systems of differential equations.

Functions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118015.png" /> for which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118016.png" /> is not a supermartingale, but from which one can readily form a supermartingale, are sometimes also called Lyapunov stochastic functions. Below typical results are presented on the qualitative behaviour of trajectories of Markov processes in terms of a Lyapunov stochastic function.

1) If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118017.png" /> is a right-continuous strong Markov process in <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118018.png" />, defined up to the instant <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118019.png" /> of first leaving an arbitrary compact set, and if there is a Lyapunov stochastic function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118020.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118021.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118022.png" />, and a constant <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118023.png" /> such that

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118024.png" /></td> </tr></table>

then

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118025.png" /></td> </tr></table>

for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118026.png" />; that is, the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118027.png" /> is defined for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118028.png" /> (is indefinitely extendable).

2) For the stationary Markov process in <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118029.png" /> corresponding to a [[Transition function|transition function]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118030.png" /> to exist it is sufficient that there should be a function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118031.png" /> for which

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118032.png" /></td> </tr></table>

as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/l/l061/l061180/l06118033.png" />.

By means of the Lyapunov stochastic function one can carry over to Markov processes the main theorems of the direct Lyapunov method; these functions have also found application in the investigation of processes in discrete time.

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  H.J. Kushner,   "Stochastic stability and control" , Acad. Press  (1967)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  R.Z. [R.Z. Khas'minskii] Has'minskii,   "Stochastic stability of differential equations" , Sijthoff &amp; Noordhoff  (1980)  (Translated from Russian)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  V.V. Kalashnikov,   "Qualitative analysis of the behaviour of complex systems by the method of test functions" , Moscow  (1978)  (In Russian)</TD></TR></table>



====Comments====
The phrase stochastic Lyapunov function is more common than  "Lyapunov stochastic function" .

Recently, stochastic Lyapunov functions have been used to prove convergence of recursive algorithms driven by stochastic processes. Convergence problems of this type arise in system identification and adaptive control.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  G.C. Goodwin,   P.J. Ramagadge,   P.E. Caines,   "Discrete time stochastic adaptive control"  ''SIAM J. Control Optim.'' , '''19'''  (1981)  pp. 829–853</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  M. Metivier,   P. Priouret,   "Applications of a Kushner and Clark lemma to general classes of stochastic algorithms"  ''IEEE Trans. Inform. Theory'' , '''30'''  (1984)  pp. 140–151</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  V. Solo,   "The convergence of AML"  ''IEEE Trans. Autom. Control'' , '''24'''  (1979)  pp. 958–962</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  J.H. van Schuppen,   "Convergence results for continuous-time stochastic filtering algorithms"  ''J. Math. Anal. Appl.'' , '''96'''  (1983)  pp. 209–225</TD></TR></table>