View source for Lyapunov stochastic function

<!--
l0611801.png
$#A+1 = 33 n = 0
$#C+1 = 33 : ~/encyclopedia/old_files/data/L061/L.0601180 Lyapunov stochastic function
Automatically converted into TeX, above some diagnostics.
Please remove this comment and the {{TEX|auto}} line below,
if TeX found to be correct.
-->

{{TEX|auto}}
{{TEX|done}}

A non-negative function  $  V ( t , x ) $
for which the pair  $  ( V ( t , X ( t) ) , F _ {t} ) $
is a supermartingale for some random process  $  X $
up to the instant  $  t $(
cf. also [[Martingale|Martingale]]). If  $  X ( t) $
is a [[Markov process|Markov process]], then the Lyapunov stochastic function is a function for which the Lyapunov stochastic operator

$$ 
LV {( t ,x ) } =
$$

$$ 
= \ 
\lim\limits _ {h \rightarrow 0 }  
\frac{1}{h}
 {\mathsf E} [ V ( t+ h ,\ 
X ( t+ h)) - V ( t, X ( t)) \mid  X ( t) = x ]
$$

is non-positive. The operator  $  L $
is the [[Infinitesimal operator|infinitesimal operator]] of the process  $  ( t , X ( t)) $, 
and so the verification of the condition  $  LV \leq  0 $
is easily carried out in specific cases. The operator  $  L $
goes into the usual Lyapunov operator  $  dV ( t , X ( t)) / dt $
when the process  $  X $
is determinate and is described by a system of differential equations. By means of the Lyapunov stochastic function it is possible to verify a number of qualitative properties of the trajectories of  $  X ( t) $; 
their role in the theory of random processes is similar to the role of the classical [[Lyapunov function|Lyapunov function]] in the theory of systems of differential equations.

Functions  $  V ( t , x ) $
for which  $  ( V ( t , X ( t)) , F _ {t} ) $
is not a supermartingale, but from which one can readily form a supermartingale, are sometimes also called Lyapunov stochastic functions. Below typical results are presented on the qualitative behaviour of trajectories of Markov processes in terms of a Lyapunov stochastic function.

1) If  $  X ( t) $
is a right-continuous strong Markov process in  $  \mathbf R  ^ {k} $, 
defined up to the instant  $  \tau $
of first leaving an arbitrary compact set, and if there is a Lyapunov stochastic function  $  V ( t , x ) $, 
$  t > 0 $, 
$  x \in \mathbf R  ^ {k} $, 
and a constant  $  c $
such that

$$ 
\inf _ {t , | x| > R }  V ( t , x )  \rightarrow  \infty \ \ 
\textrm{ as }  R \rightarrow \infty ,\ \ 
L V  \leq   c V ,
$$

then

$$ 
{\mathsf P} \{ \tau < \infty \mid  X ( 0) = x \}  =  1
$$

for any  $  x \in \mathbf R  ^ {k} $; 
that is, the process  $  X $
is defined for all  $  t > 0 $(
is indefinitely extendable).

2) For the stationary Markov process in  $  \mathbf R  ^ {k} $
corresponding to a [[Transition function|transition function]]  $  P ( t , x , A ) $
to exist it is sufficient that there should be a function  $  V ( x) \geq  0 $
for which

$$ 
\sup _ {| x| > R }  L V ( x)  \rightarrow  - \infty
$$

as  $  R \rightarrow \infty $.

By means of the Lyapunov stochastic function one can carry over to Markov processes the main theorems of the direct Lyapunov method; these functions have also found application in the investigation of processes in discrete time.

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  H.J. Kushner,   "Stochastic stability and control" , Acad. Press  (1967)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  R.Z. [R.Z. Khas'minskii] Has'minskii,   "Stochastic stability of differential equations" , Sijthoff &amp; Noordhoff  (1980)  (Translated from Russian)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  V.V. Kalashnikov,   "Qualitative analysis of the behaviour of complex systems by the method of test functions" , Moscow  (1978)  (In Russian)</TD></TR></table>

====Comments====
The phrase stochastic Lyapunov function is more common than  "Lyapunov stochastic function" .

Recently, stochastic Lyapunov functions have been used to prove convergence of recursive algorithms driven by stochastic processes. Convergence problems of this type arise in system identification and adaptive control.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  G.C. Goodwin,   P.J. Ramagadge,   P.E. Caines,   "Discrete time stochastic adaptive control"  ''SIAM J. Control Optim.'' , '''19'''  (1981)  pp. 829–853</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  M. Metivier,   P. Priouret,   "Applications of a Kushner and Clark lemma to general classes of stochastic algorithms"  ''IEEE Trans. Inform. Theory'' , '''30'''  (1984)  pp. 140–151</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  V. Solo,   "The convergence of AML"  ''IEEE Trans. Autom. Control'' , '''24'''  (1979)  pp. 958–962</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  J.H. van Schuppen,   "Convergence results for continuous-time stochastic filtering algorithms"  ''J. Math. Anal. Appl.'' , '''96'''  (1983)  pp. 209–225</TD></TR></table>