Namespaces
Variants
Actions

Difference between revisions of "Stationary stochastic process"

From Encyclopedia of Mathematics
Jump to: navigation, search
(refs format)
m (tex encoded by computer)
Line 1: Line 1:
 +
<!--
 +
s0872801.png
 +
$#A+1 = 194 n = 0
 +
$#C+1 = 194 : ~/encyclopedia/old_files/data/S087/S.0807280 Stationary stochastic process,
 +
Automatically converted into TeX, above some diagnostics.
 +
Please remove this comment and the {{TEX|auto}} line below,
 +
if TeX found to be correct.
 +
-->
 +
 +
{{TEX|auto}}
 +
{{TEX|done}}
 +
 
''stochastic process, homogeneous in time''
 
''stochastic process, homogeneous in time''
  
Line 5: Line 17:
 
[[Category:Stochastic processes]]
 
[[Category:Stochastic processes]]
  
A [[Stochastic process|stochastic process]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872801.png" /> whose statistical characteristics do not change in the course of time <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872802.png" />, i.e. are invariant relative to translations in time: <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872803.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872804.png" />, for any fixed value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872805.png" /> (either a real number or an integer, depending on whether one is dealing with a stochastic process in continuous or in discrete time). The concept of a stationary stochastic process is widely used in applications of probability theory in various areas of natural science and technology, since these processes accurately describe many real phenomena accompanied by unordered fluctuations. For example, the pulsations of the force of a current or the voltage in an electrical chain (electrical "noise" ) can be considered as stationary stochastic processes if the chain is in a stationary system; the pulsations of velocity or pressure at a point of a turbulent flow are stationary stochastic processes if the flow is stationary, etc.
+
A [[Stochastic process|stochastic process]] $  X( t) $
 +
whose statistical characteristics do not change in the course of time $  t $,  
 +
i.e. are invariant relative to translations in time: $  t \rightarrow t + a $,  
 +
$  X( t) \rightarrow X( t+ a) $,  
 +
for any fixed value of $  a $(
 +
either a real number or an integer, depending on whether one is dealing with a stochastic process in continuous or in discrete time). The concept of a stationary stochastic process is widely used in applications of probability theory in various areas of natural science and technology, since these processes accurately describe many real phenomena accompanied by unordered fluctuations. For example, the pulsations of the force of a current or the voltage in an electrical chain (electrical "noise" ) can be considered as stationary stochastic processes if the chain is in a stationary system; the pulsations of velocity or pressure at a point of a turbulent flow are stationary stochastic processes if the flow is stationary, etc.
  
In the mathematical theory of stationary stochastic processes, an important role is played by the moments of the probability distribution of the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872806.png" />, and especially by the moments of the first two orders — the mean value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872807.png" />, and its covariance function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872808.png" />, or, equivalently, the correlation function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s0872809.png" />. In much of the research into the theory of stationary stochastic processes, the properties that are completely defined by the characteristics <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728010.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728011.png" /> alone are studied (the so-called correlation theory or theory of second-order stationary stochastic processes). Accordingly, the stochastic processes <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728012.png" /> for which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728013.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728014.png" /> do not depend on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728015.png" /> are often separated into a special class and are called stationary stochastic processes in the wide sense. The more special stochastic processes, none of whose characteristics change with time (so that the distribution function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728016.png" /> of an <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728017.png" />-dimensional random variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728018.png" /> depends here, for any <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728019.png" />, only on the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728020.png" /> differences <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728021.png" />) are called stationary stochastic processes in the strict sense. Accordingly, the theory of stationary stochastic processes is divided into the theory of stationary stochastic processes in the strict sense and in the wide sense, with a different mathematical apparatus used in each.
+
In the mathematical theory of stationary stochastic processes, an important role is played by the moments of the probability distribution of the process $  X( t) $,  
 +
and especially by the moments of the first two orders — the mean value $  {\mathsf E} X( t) = m $,  
 +
and its covariance function $  {\mathsf E} [( X( t + \tau ) - {\mathsf E} X( t + \tau )) ( X( t) - EX( t)) ] $,  
 +
or, equivalently, the correlation function $  E X( t+ \tau ) X( t) = B( \tau ) $.  
 +
In much of the research into the theory of stationary stochastic processes, the properties that are completely defined by the characteristics $  m $
 +
and $  B( \tau ) $
 +
alone are studied (the so-called correlation theory or theory of second-order stationary stochastic processes). Accordingly, the stochastic processes $  X( t) $
 +
for which $  {\mathsf E} X( t) $
 +
and $  {\mathsf E} X( t + \tau ) X( t) $
 +
do not depend on $  t $
 +
are often separated into a special class and are called stationary stochastic processes in the wide sense. The more special stochastic processes, none of whose characteristics change with time (so that the distribution function $  F _ {t _ {1}  \dots t _ {n} } ( x _ {1} \dots x _ {n} ) $
 +
of an $  n $-
 +
dimensional random variable $  \{ X( t _ {1} ) \dots X( t _ {n} ) \} $
 +
depends here, for any $  n $,  
 +
only on the $  n- 1 $
 +
differences $  t _ {2} - t _ {1} \dots t _ {n} - t _ {1} $)  
 +
are called stationary stochastic processes in the strict sense. Accordingly, the theory of stationary stochastic processes is divided into the theory of stationary stochastic processes in the strict sense and in the wide sense, with a different mathematical apparatus used in each.
  
In the strict sense, the theory can be stated outside the framework of probability theory as the theory of one-parameter groups of transformations of a measure space that preserve the measure; this theory is very close to the general theory of dynamical systems (cf. [[Dynamical system|Dynamical system]]) and to [[Ergodic theory|ergodic theory]]. The most important general theorem of the theory of stationary stochastic processes in the strict sense is the Birkhoff–Khinchin ergodic theorem, according to which for any stationary stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728022.png" /> in the strict sense having a mathematical expectation (i.e. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728023.png" />), the limit
+
In the strict sense, the theory can be stated outside the framework of probability theory as the theory of one-parameter groups of transformations of a measure space that preserve the measure; this theory is very close to the general theory of dynamical systems (cf. [[Dynamical system|Dynamical system]]) and to [[Ergodic theory|ergodic theory]]. The most important general theorem of the theory of stationary stochastic processes in the strict sense is the Birkhoff–Khinchin ergodic theorem, according to which for any stationary stochastic process $  X( t) $
 +
in the strict sense having a mathematical expectation (i.e. $  {\mathsf E} | X( t) | < \infty $),  
 +
the limit
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728024.png" /></td> <td valign="top" style="width:5%;text-align:right;">(1)</td></tr></table>
+
$$ \tag{1 }
 +
\lim\limits _ {T- S \rightarrow \infty } 
 +
\frac{1}{T-}
 +
S \int\limits _ { S } ^ { T }  X( t) dt  =  \widehat{X}  ,
 +
$$
  
 
or
 
or
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728025.png" /></td> <td valign="top" style="width:5%;text-align:right;">(1a)</td></tr></table>
+
$$ \tag{1a }
 +
\lim\limits _ {T- S \rightarrow \infty } 
 +
\frac{1}{T-}
 +
S \sum _ { t= } S+ 1 ^ { T }  X( t) =  \widehat{X}  ,
 +
$$
  
exists with probability 1 (formula (1) relates to processes in continuous time, while (1a) relates to processes in discrete time). There is a result of E.E. Slutskii {{Cite|Sl}}, related to stationary stochastic processes in the wide sense, which states that the limit (1) or (1a) exists in mean square. This limit coincides with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728026.png" /> if and only if
+
exists with probability 1 (formula (1) relates to processes in continuous time, while (1a) relates to processes in discrete time). There is a result of E.E. Slutskii {{Cite|Sl}}, related to stationary stochastic processes in the wide sense, which states that the limit (1) or (1a) exists in mean square. This limit coincides with $  {\mathsf E} X( t) $
 +
if and only if
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728027.png" /></td> <td valign="top" style="width:5%;text-align:right;">(2)</td></tr></table>
+
$$ \tag{2 }
 +
\lim\limits _ {T \rightarrow \infty }  T  ^ {-} 1 \int\limits _ { 0 } ^ { T }  b( \tau ) d \tau  =  0 ,
 +
$$
  
 
or
 
or
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728028.png" /></td> <td valign="top" style="width:5%;text-align:right;">(2a)</td></tr></table>
+
$$ \tag{2a }
 +
\lim\limits _ {T \rightarrow \infty } \
 +
T  ^ {-} 1 \sum _ {\tau = 0 } ^ { T- }  1 b( \tau ) =  0,
 +
$$
  
 
where
 
where
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728029.png" /></td> </tr></table>
+
$$
 +
b( \tau )  = B( \tau ) - m  ^ {2}  = \
 +
{\mathsf E} [ X( t+ \tau ) - m][ X( t) - m]
 +
$$
  
(the von Neumann (<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728031.png" />-) ergodic theorem). These conditions are satisfied, in particular, when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728033.png" /> as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728034.png" />. The Birkhoff–Khinchin theorem can be applied to various stationary stochastic processes in the strict sense of the form
+
(the von Neumann ( $  L _ {2} $-)  
 +
ergodic theorem). These conditions are satisfied, in particular, when $  b( \tau ) \rightarrow 0 $
 +
as $  \tau \rightarrow \infty $.  
 +
The Birkhoff–Khinchin theorem can be applied to various stationary stochastic processes in the strict sense of the form
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728035.png" /></td> </tr></table>
+
$$
 +
Y _  \Phi  ( s)  = \Phi [ X( t+ s)],
 +
$$
  
where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728036.png" /> is an arbitrary functional of the stationary stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728037.png" />, and is a random variable which has a mathematical expectation; if for all such stationary stochastic processes <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728038.png" /> the corresponding limit <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728039.png" /> coincides with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728040.png" />, then <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728041.png" /> is called a metrically transitive stationary stochastic process. For stationary Gaussian stochastic processes <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728042.png" />, the condition of being stationary in the strict sense coincides with the condition of being stationary in the wide sense; metric transitivity will occur if and only if the [[Spectral function|spectral function]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728043.png" /> of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728044.png" /> is a continuous function of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728045.png" /> (see, for example, {{Cite|R}}, {{Cite|CL}}). There are, in general, no simple necessary and sufficient conditions for the metric transitivity of a stationary stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728046.png" />.
+
where $  \Phi [ X( t)] $
 +
is an arbitrary functional of the stationary stochastic process $  X( t) $,  
 +
and is a random variable which has a mathematical expectation; if for all such stationary stochastic processes $  Y _  \Phi  ( s) $
 +
the corresponding limit $  \widehat{Y}  _  \Phi  $
 +
coincides with $  {\mathsf E} Y _  \Phi  $,  
 +
then $  X( t) $
 +
is called a metrically transitive stationary stochastic process. For stationary Gaussian stochastic processes $  X( t) $,  
 +
the condition of being stationary in the strict sense coincides with the condition of being stationary in the wide sense; metric transitivity will occur if and only if the [[Spectral function|spectral function]] $  F( \lambda ) $
 +
of $  X( t) $
 +
is a continuous function of $  \lambda $(
 +
see, for example, {{Cite|R}}, {{Cite|CL}}). There are, in general, no simple necessary and sufficient conditions for the metric transitivity of a stationary stochastic process $  X( t) $.
  
Apart from the above result relating to metric transitivity, there are also other results specifically for stationary Gaussian stochastic processes. For these processes, detailed studies have been made of the question of the local properties of the realizations of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728047.png" /> (i.e. of individual observed values), and of the statistical properties of the sequence of zeros or maxima of the realizations of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728048.png" />, and the points of intersection of it with a given level (see, for example, {{Cite|CL}}). A typical example of the results related to intersections with a level is the statement that, given broad regularity conditions, the set of points of intersection of a high level <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728049.png" /> with the stationary Gaussian stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728050.png" /> in a certain special time scale (dependent on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728051.png" /> and tending rapidly to infinity when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728052.png" />) converges to a Poisson flow of events of unit intensity when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728053.png" /> (see {{Cite|CL}}).
+
Apart from the above result relating to metric transitivity, there are also other results specifically for stationary Gaussian stochastic processes. For these processes, detailed studies have been made of the question of the local properties of the realizations of $  X( t) $(
 +
i.e. of individual observed values), and of the statistical properties of the sequence of zeros or maxima of the realizations of $  X( t) $,  
 +
and the points of intersection of it with a given level (see, for example, {{Cite|CL}}). A typical example of the results related to intersections with a level is the statement that, given broad regularity conditions, the set of points of intersection of a high level $  x= u $
 +
with the stationary Gaussian stochastic process $  X( t) $
 +
in a certain special time scale (dependent on $  u $
 +
and tending rapidly to infinity when $  u \rightarrow \infty $)  
 +
converges to a Poisson flow of events of unit intensity when $  u \rightarrow \infty $(
 +
see {{Cite|CL}}).
  
When studying stationary stochastic processes in the wide sense, the Hilbert space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728054.png" /> of linear combinations of values of the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728055.png" /> and the mean-square limits of sequences of such linear combinations are examined, and a scalar product is defined in it by the formula <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728056.png" />. In this case, the transformation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728057.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728058.png" /> is a fixed number, will generate a linear unitary operator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728059.png" /> mapping the space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728060.png" /> onto itself; the family of operators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728061.png" /> clearly satisfies the condition <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728062.png" />, while the values <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728063.png" /> form a set of points (a curve if the time <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728064.png" /> is continuous, and a countable sequence of points if the time is discrete), mapped onto itself by all operators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728065.png" />. Accordingly, the theory of stationary stochastic processes in the wide sense can be reformulated in terms of functional analysis as the study of the sets of points <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728066.png" /> of the Hilbert space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728067.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728068.png" /> is a family of linear unitary operators such that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728069.png" /> (cf. also [[Semi-group of operators|Semi-group of operators]]).
+
When studying stationary stochastic processes in the wide sense, the Hilbert space $  H _ {X} $
 +
of linear combinations of values of the process $  X( t) $
 +
and the mean-square limits of sequences of such linear combinations are examined, and a scalar product is defined in it by the formula $  ( Y _ {1} , Y _ {2} ) = {\mathsf E} Y _ {1} Y _ {2} $.  
 +
In this case, the transformation $  X( t) \rightarrow X( t+ a) $,  
 +
where $  a $
 +
is a fixed number, will generate a linear unitary operator $  U _ {a} $
 +
mapping the space $  H _ {X} $
 +
onto itself; the family of operators $  U _ {a} $
 +
clearly satisfies the condition $  U _ {a} U _ {b} = U _ {a+} b $,  
 +
while the values $  X( t) = U _ {t} X( 0) $
 +
form a set of points (a curve if the time $  t $
 +
is continuous, and a countable sequence of points if the time is discrete), mapped onto itself by all operators $  U _ {a} $.  
 +
Accordingly, the theory of stationary stochastic processes in the wide sense can be reformulated in terms of functional analysis as the study of the sets of points $  X( t) = U _ {t} X( 0) $
 +
of the Hilbert space $  H _ {X} $,  
 +
where $  U _ {t} $
 +
is a family of linear unitary operators such that $  U _ {a} U _ {b} = U _ {a+} b $(
 +
cf. also [[Semi-group of operators|Semi-group of operators]]).
  
Spectral considerations, based on the expansion of the stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728070.png" /> and its correlation function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728071.png" /> into a Fourier–Stieltjes integral, are central to the theory of stationary stochastic processes in the wide sense. By Khinchin's theorem {{Cite|Kh}} (which is a simple consequence of Bochner's analytic theorem on the general form of a positive-definite function), the correlation function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728072.png" /> of a stationary stochastic process in continuous time can always be represented in the form
+
Spectral considerations, based on the expansion of the stochastic process $  X( t) $
 +
and its correlation function $  B( \tau ) $
 +
into a Fourier–Stieltjes integral, are central to the theory of stationary stochastic processes in the wide sense. By Khinchin's theorem {{Cite|Kh}} (which is a simple consequence of Bochner's analytic theorem on the general form of a positive-definite function), the correlation function $  B( \tau ) $
 +
of a stationary stochastic process in continuous time can always be represented in the form
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728073.png" /></td> <td valign="top" style="width:5%;text-align:right;">(3)</td></tr></table>
+
$$ \tag{3 }
 +
B( \tau )  = \int\limits _  \Lambda  e ^ {i \tau \lambda }  dF( \lambda ),
 +
$$
  
where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728074.png" /> is a bounded monotone non-decreasing function of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728075.png" />, while <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728076.png" />; the Herglotz theorem on the general form of positive-definite sequences similarly shows that the same representation, but with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728077.png" />, also holds for the correlation function of a stationary stochastic process in discrete time. If the correlation function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728078.png" /> decreases sufficiently rapidly as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728079.png" /> (as is most often the case in applications under the condition that by <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728080.png" /> one understands the difference <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728081.png" />, i.e. it is considered that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728082.png" />), then the integral at the right-hand side of (3) becomes an ordinary Fourier integral
+
where $  F( \lambda ) $
 +
is a bounded monotone non-decreasing function of $  \lambda $,  
 +
while $  \Lambda = (- \infty , \infty ) $;  
 +
the Herglotz theorem on the general form of positive-definite sequences similarly shows that the same representation, but with $  \Lambda = [- \pi , \pi ] $,  
 +
also holds for the correlation function of a stationary stochastic process in discrete time. If the correlation function $  B( \tau ) $
 +
decreases sufficiently rapidly as $  | \tau | \rightarrow \infty $(
 +
as is most often the case in applications under the condition that by $  X( t) $
 +
one understands the difference $  X( t)- m $,  
 +
i.e. it is considered that $  {\mathsf E} X( t) = 0 $),  
 +
then the integral at the right-hand side of (3) becomes an ordinary Fourier integral
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728083.png" /></td> <td valign="top" style="width:5%;text-align:right;">(4)</td></tr></table>
+
$$ \tag{4 }
 +
B( \tau )  = \int\limits _  \Lambda  e ^ {i \tau \lambda } f( \lambda ) d \lambda ,
 +
$$
  
where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728084.png" /> is a non-negative function. The function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728085.png" /> is called the spectral function of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728086.png" />, while the function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728087.png" /> (in cases where the equality (4) holds) is called its spectral density. Starting from the Khinchin formula (3) (or from the definition of the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728088.png" /> in the form of the set of points <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728089.png" /> in the Hilbert space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728090.png" />, and Stone's theorem on the spectral representation of one-parameter groups of unitary operators in a Hilbert space), it can also be demonstrated that the process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728091.png" /> itself permits a spectral decomposition in the form
+
where $  f( \lambda ) = F ^ { \prime } ( \lambda ) $
 +
is a non-negative function. The function $  F( \lambda ) $
 +
is called the spectral function of $  X( t) $,  
 +
while the function $  f( \lambda ) $(
 +
in cases where the equality (4) holds) is called its spectral density. Starting from the Khinchin formula (3) (or from the definition of the process $  X( t) $
 +
in the form of the set of points $  X( t) = U _ {t} X( 0) $
 +
in the Hilbert space $  H _ {X} $,  
 +
and Stone's theorem on the spectral representation of one-parameter groups of unitary operators in a Hilbert space), it can also be demonstrated that the process $  X( t) $
 +
itself permits a spectral decomposition in the form
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728092.png" /></td> <td valign="top" style="width:5%;text-align:right;">(5)</td></tr></table>
+
$$ \tag{5 }
 +
X( t)  = \int\limits _  \Lambda  e ^ {it \lambda }  dZ( \lambda ),
 +
$$
  
where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728093.png" /> is a random function with uncorrelated increments (i.e. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728094.png" /> when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728095.png" />) which satisfies the condition <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728096.png" />, while the integral at the right-hand side is understood to be the mean-square limit of the corresponding sequence of integral sums. The decomposition (5) provides grounds for considering any stationary stochastic process in the wide sense as a superposition of a set of non-correlated harmonic oscillations of different frequencies with random amplitudes and phases; the spectral function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728097.png" /> and the spectral density <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728098.png" /> define the distribution of the average energy (or, more accurately, of the power) of the harmonic oscillations with frequency spectrum <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s08728099.png" /> that constitute <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280100.png" /> (as a result of which the function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280101.png" /> in applied research is often called the energy spectrum, or power spectrum, of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280102.png" />).
+
where $  Z( \lambda ) $
 +
is a random function with uncorrelated increments (i.e. $  {\mathsf E} dZ( \lambda _ {1} ) dZ( \lambda _ {2} ) = 0 $
 +
when $  \lambda _ {1} \neq \lambda _ {2} $)  
 +
which satisfies the condition $  {\mathsf E} | dZ( \lambda ) |  ^ {2} = dF( \lambda ) $,  
 +
while the integral at the right-hand side is understood to be the mean-square limit of the corresponding sequence of integral sums. The decomposition (5) provides grounds for considering any stationary stochastic process in the wide sense as a superposition of a set of non-correlated harmonic oscillations of different frequencies with random amplitudes and phases; the spectral function $  F( \lambda ) $
 +
and the spectral density $  f( \lambda ) $
 +
define the distribution of the average energy (or, more accurately, of the power) of the harmonic oscillations with frequency spectrum $  \lambda $
 +
that constitute $  X( t) $(
 +
as a result of which the function $  f( \lambda ) $
 +
in applied research is often called the energy spectrum, or power spectrum, of $  X( t) $).
  
The spectral decomposition of the correlation function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280103.png" />, defined by formula (3), demonstrates that the mapping <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280104.png" />, which maps elements <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280105.png" /> of the Hilbert space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280106.png" /> to elements <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280107.png" /> of the Hilbert space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280108.png" /> of complex-valued functions on the set <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280109.png" /> with a modulus, square-integrable with respect to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280110.png" />, is an isometric mapping of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280111.png" /> into <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280112.png" />. This mapping can be extended to an isometric linear mapping <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280113.png" /> of the whole space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280114.png" /> onto the space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280115.png" />, a fact that allows one to reformulate many problems in the theory of stationary stochastic processes in the wide sense as problems in function theory.
+
The spectral decomposition of the correlation function $  B( \tau ) $,  
 +
defined by formula (3), demonstrates that the mapping $  X( t) \rightarrow e ^ {it \lambda } $,  
 +
which maps elements $  X( t) $
 +
of the Hilbert space $  H _ {X} $
 +
to elements $  e ^ {it \lambda } $
 +
of the Hilbert space $  L _ {2} ( dF  ) $
 +
of complex-valued functions on the set $  \Lambda $
 +
with a modulus, square-integrable with respect to $  dF( \lambda ) $,  
 +
is an isometric mapping of $  H $
 +
into $  L _ {2} ( dF  ) $.  
 +
This mapping can be extended to an isometric linear mapping $  M $
 +
of the whole space $  H _ {X} $
 +
onto the space $  L _ {2} ( dF  ) $,  
 +
a fact that allows one to reformulate many problems in the theory of stationary stochastic processes in the wide sense as problems in function theory.
  
A significant part of the theory of stationary stochastic processes in the wide sense is devoted to methods of solving linear approximation problems for such processes, i.e. methods of locating a linear combination of any "known" values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280116.png" /> that best approximates (in the sense of the minimum least-square error) a certain "unknown" value of the same process or any "unknown" random variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280117.png" />. In particular, the problem of optimal linear extrapolation of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280118.png" /> consists of finding the best approximation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280119.png" /> of the value <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280120.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280121.png" />, that linearly depends on the "past values" of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280122.png" /> with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280123.png" />; the problem of optimal linear interpolation consists of finding the best approximation for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280124.png" /> that linearly depends on the values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280125.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280126.png" /> runs through all values that do not belong to a specific interval of the time axis (to which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280127.png" /> does belong); the problem of optimal linear filtering can be formulated as the problem of finding the best approximation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280128.png" /> for a certain random variable <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280129.png" /> (which is usually the value for some <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280130.png" /> of a stationary stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280131.png" />, correlated with <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280132.png" />, whereby <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280133.png" /> most often plays the part of a "signal" , while <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280134.png" /> is the sum of the "signal" and a "noise" <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280135.png" /> that interferes with it, and the sum is known from the observations) that linearly depends on the values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280136.png" /> when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280137.png" /> (see [[Stochastic processes, prediction of|Stochastic processes, prediction of]]; [[Stochastic processes, filtering of|Stochastic processes, filtering of]]; [[Stochastic processes, interpolation of|Stochastic processes, interpolation of]]).
+
A significant part of the theory of stationary stochastic processes in the wide sense is devoted to methods of solving linear approximation problems for such processes, i.e. methods of locating a linear combination of any "known" values of $  X( t) $
 +
that best approximates (in the sense of the minimum least-square error) a certain "unknown" value of the same process or any "unknown" random variable $  Y $.  
 +
In particular, the problem of optimal linear extrapolation of $  X( t) $
 +
consists of finding the best approximation $  X  ^ {*} ( s) $
 +
of the value $  X( s) $,  
 +
s > 0 $,  
 +
that linearly depends on the "past values" of $  X( t) $
 +
with $  t \leq  0 $;  
 +
the problem of optimal linear interpolation consists of finding the best approximation for $  X( s) $
 +
that linearly depends on the values of $  X( t) $,  
 +
where $  t $
 +
runs through all values that do not belong to a specific interval of the time axis (to which s $
 +
does belong); the problem of optimal linear filtering can be formulated as the problem of finding the best approximation $  Y  ^ {*} $
 +
for a certain random variable $  Y $(
 +
which is usually the value for some $  t= s $
 +
of a stationary stochastic process $  Y( t) $,  
 +
correlated with $  X( t) $,  
 +
whereby $  Y( t) $
 +
most often plays the part of a "signal" , while $  X( t) = Y( t) + N( t) $
 +
is the sum of the "signal" and a "noise" $  N( t) $
 +
that interferes with it, and the sum is known from the observations) that linearly depends on the values of $  X( t) $
 +
when $  t \leq  0 $(
 +
see [[Stochastic processes, prediction of|Stochastic processes, prediction of]]; [[Stochastic processes, filtering of|Stochastic processes, filtering of]]; [[Stochastic processes, interpolation of|Stochastic processes, interpolation of]]).
  
All these problems reduce geometrically to the problem of projecting a point of the Hilbert space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280138.png" /> (or of its extension) orthogonally onto a given subspace of this space. Relying on this geometric interpretation and on the isomorphism of the spaces <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280139.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280140.png" />, A.N. Kolmogorov has deduced general formulas that make it possible to determine the mean-square error of optimal linear extrapolation or interpolation, corresponding to the case where the value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280141.png" /> is unknown only when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280142.png" />, by means of the spectral function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280143.png" /> of the stationary stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280144.png" /> in discrete time <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280145.png" /> (see {{Cite|R}}, {{Cite|Ko}}–{{Cite|D}}). When used for the extrapolation problem, the same results were obtained for processes <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280146.png" /> in continuous time by M.G. Krein and K. Karhunen. N. Wiener {{Cite|W}} demonstrated that the search for the best approximation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280147.png" /> or <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280148.png" /> in the case of problems of optimal linear extrapolation and filtering can be reduced to the solution of a certain integral equation of Wiener–Hopf type, or (when <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280149.png" /> is discrete) of the discrete analogue of such an equation, which makes it possible to use the factorization method (see [[Wiener–Hopf equation|Wiener–Hopf equation]]; [[Wiener–Hopf method|Wiener–Hopf method]]). Problems of optimal linear extrapolation or filtering of a stationary stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280150.png" /> in continuous time in the case where not all its past values for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280151.png" /> are known but only its values on a finite interval <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280152.png" />, as well as the problem of optimal linear interpolation of such an <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280153.png" />, can be reduced to certain problems of establishing a special form of differential equation (a "generalized string equation" ) by means of its spectrum (see {{Cite|Kr}}, {{Cite|DM}}).
+
All these problems reduce geometrically to the problem of projecting a point of the Hilbert space $  H _ {X} $(
 +
or of its extension) orthogonally onto a given subspace of this space. Relying on this geometric interpretation and on the isomorphism of the spaces $  H _ {X} $
 +
and $  L _ {2} ( dF  ) $,  
 +
A.N. Kolmogorov has deduced general formulas that make it possible to determine the mean-square error of optimal linear extrapolation or interpolation, corresponding to the case where the value of $  X( t) $
 +
is unknown only when $  t= s $,  
 +
by means of the spectral function $  F( \lambda ) $
 +
of the stationary stochastic process $  X( t) $
 +
in discrete time $  t $(
 +
see {{Cite|R}}, {{Cite|Ko}}–{{Cite|D}}). When used for the extrapolation problem, the same results were obtained for processes $  X( t) $
 +
in continuous time by M.G. Krein and K. Karhunen. N. Wiener {{Cite|W}} demonstrated that the search for the best approximation $  X  ^ {*} ( s) $
 +
or $  Y  ^ {*} = Y  ^ {*} ( s) $
 +
in the case of problems of optimal linear extrapolation and filtering can be reduced to the solution of a certain integral equation of Wiener–Hopf type, or (when $  t $
 +
is discrete) of the discrete analogue of such an equation, which makes it possible to use the factorization method (see [[Wiener–Hopf equation|Wiener–Hopf equation]]; [[Wiener–Hopf method|Wiener–Hopf method]]). Problems of optimal linear extrapolation or filtering of a stationary stochastic process $  X( t) $
 +
in continuous time in the case where not all its past values for $  t \leq  0 $
 +
are known but only its values on a finite interval $  - T \leq  t \leq  0 $,  
 +
as well as the problem of optimal linear interpolation of such an $  X( t) $,  
 +
can be reduced to certain problems of establishing a special form of differential equation (a "generalized string equation" ) by means of its spectrum (see {{Cite|Kr}}, {{Cite|DM}}).
  
The above approaches to the solution of problems of optimal linear extrapolation, interpolation and filtering provide sufficiently-simple explicit formulas for the required best approximation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280154.png" /> or <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280155.png" /> that can be successfully used in practice only in certain exceptional cases. One important case in which such explicit formulas do exist is the case of a stationary stochastic process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280156.png" /> with rational spectral density <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280157.png" /> relative to the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280158.png" /> (if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280159.png" /> is discrete) or relative to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280160.png" /> (if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280161.png" /> is continuous), which was studied in detail by Wiener {{Cite|W}} (for applications to problems of extrapolation and filtering by values where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280162.png" />). It was subsequently demonstrated that for such stationary stochastic processes with rational spectral density there is also an explicit solution of the problems of linear interpolation, extrapolation and filtering by means of data on a finite interval <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280163.png" /> (see, for example, {{Cite|R}}, {{Cite|Y}}). The simplicity of processes with rational spectral density can be explained by the fact that such stationary stochastic processes (and practically only they) are a one-dimensional component of a multi-dimensional stationary [[Markov process|Markov process]] (see {{Cite|D2}}).
+
The above approaches to the solution of problems of optimal linear extrapolation, interpolation and filtering provide sufficiently-simple explicit formulas for the required best approximation $  X  ^ {*} ( s) $
 +
or $  Y  ^ {*} $
 +
that can be successfully used in practice only in certain exceptional cases. One important case in which such explicit formulas do exist is the case of a stationary stochastic process $  X( t) $
 +
with rational spectral density $  f( \lambda ) $
 +
relative to the $  e ^ {i \lambda } $(
 +
if $  t $
 +
is discrete) or relative to $  \lambda $(
 +
if $  t $
 +
is continuous), which was studied in detail by Wiener {{Cite|W}} (for applications to problems of extrapolation and filtering by values where $  t \leq  0 $).  
 +
It was subsequently demonstrated that for such stationary stochastic processes with rational spectral density there is also an explicit solution of the problems of linear interpolation, extrapolation and filtering by means of data on a finite interval $  - T \leq  t \leq  0 $(
 +
see, for example, {{Cite|R}}, {{Cite|Y}}). The simplicity of processes with rational spectral density can be explained by the fact that such stationary stochastic processes (and practically only they) are a one-dimensional component of a multi-dimensional stationary [[Markov process|Markov process]] (see {{Cite|D2}}).
  
The concept of a stationary stochastic process permits a whole series of generalizations. One of these is the concept of a generalized stationary stochastic process. This is a generalized stochastic process (cf. [[Stochastic process, generalized|Stochastic process, generalized]]) <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280164.png" /> (i.e. a random linear functional, defined on the space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280165.png" /> of infinitely-differentiable functions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280166.png" /> with compact support), such that either the distribution function of the random vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280167.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280168.png" /> for any positive integer <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280169.png" />, real number <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280170.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280171.png" />, coincides with the probability distribution of the vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280172.png" /> (a generalized stationary stochastic process in the strict sense), or else
+
The concept of a stationary stochastic process permits a whole series of generalizations. One of these is the concept of a generalized stationary stochastic process. This is a generalized stochastic process (cf. [[Stochastic process, generalized|Stochastic process, generalized]]) $  X( \phi ) $(
 +
i.e. a random linear functional, defined on the space $  D $
 +
of infinitely-differentiable functions $  \phi ( t) $
 +
with compact support), such that either the distribution function of the random vector $  \{ X( V _ {a} \phi _ {1} ) \dots X( V _ {a} \phi _ {n} ) \} $,  
 +
where $  V _ {a} \phi ( t) = \phi ( t+ a) $
 +
for any positive integer $  n $,  
 +
real number $  a $
 +
and $  \phi _ {1} \dots \phi _ {2} \in D $,  
 +
coincides with the probability distribution of the vector $  \{ X( \phi _ {1} ) \dots X( \phi _ {n} ) \} $(
 +
a generalized stationary stochastic process in the strict sense), or else
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280173.png" /></td> </tr></table>
+
$$
 +
{\mathsf E} X( \phi )  = {\mathsf E} X( V _ {a} \phi ),
 +
$$
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280174.png" /></td> </tr></table>
+
$$
 +
{\mathsf E} X( V _ {a} \phi _ {1} ) \overline{ {X( V _ {a} \phi _ {2} ) }}\; = {\mathsf E} X( \phi _ {1} ) \overline{ {X( \phi _ {2} ) }}\; ,
 +
$$
  
for all <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280175.png" /> (a generalized stationary stochastic process in the wide sense). A generalized stationary stochastic process in the wide sense <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280176.png" /> and its correlation functional <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280177.png" /> (or covariance functional <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280178.png" />) permit a spectral decomposition related to (2) and (5) (see [[Spectral decomposition of a random function|Spectral decomposition of a random function]]). Other frequently-used generalizations of the concept of a stationary stochastic process are the concepts of a stochastic process with stationary increments of a certain order and of a homogeneous random field (cf. [[Random field, homogeneous|Random field, homogeneous]]).
+
for all $  a $(
 +
a generalized stationary stochastic process in the wide sense). A generalized stationary stochastic process in the wide sense $  X( \phi ) $
 +
and its correlation functional $  B( \phi _ {1} , \phi _ {2} ) = {\mathsf E} X( \phi _ {1} ) \overline{ {X( \phi _ {2} ) }}\; $(
 +
or covariance functional $  {\mathsf E} [ ( X( \phi _ {1} ) - {\mathsf E} X( \phi _ {1} )) \overline{ {( X( \phi _ {2} ) - {\mathsf E} X( \phi _ {2} )) }}\; ] $)  
 +
permit a spectral decomposition related to (2) and (5) (see [[Spectral decomposition of a random function|Spectral decomposition of a random function]]). Other frequently-used generalizations of the concept of a stationary stochastic process are the concepts of a stochastic process with stationary increments of a certain order and of a homogeneous random field (cf. [[Random field, homogeneous|Random field, homogeneous]]).
  
 
====References====
 
====References====
Line 99: Line 302:
 
In the English language literature one says ergodic stationary process rather than metrically-transitive stationary process. See also [[Gaussian process|Gaussian process]].
 
In the English language literature one says ergodic stationary process rather than metrically-transitive stationary process. See also [[Gaussian process|Gaussian process]].
  
The terminology concerning the phrases "correlation function" and "covariance function" is not yet completely standardized. In probability theory the following terminology seems just about universally adopted. Given two random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280179.png" /> their [[Covariance|covariance]] is <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280180.png" />, their [[Correlation coefficient|correlation coefficient]] is <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280181.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280182.png" /> is the variance of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280183.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280184.png" /> (cf. [[Dispersion|Dispersion]]), and there is no special name for the mixed second-order moment <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280185.png" />. Correspondly one has the terms covariance function and correlation function, also termed auto-correlation function and auto-covariance function, for the following quantities associated to a stochastic process:
+
The terminology concerning the phrases "correlation function" and "covariance function" is not yet completely standardized. In probability theory the following terminology seems just about universally adopted. Given two random variables $  X, Y $
 +
their [[Covariance|covariance]] is $  \mathop{\rm cov} ( X, Y) = {\mathsf E} [( X- {\mathsf E} X )( Y- {\mathsf E} Y )] $,  
 +
their [[Correlation coefficient|correlation coefficient]] is $  \rho ( X, Y) = \mathop{\rm cov} ( X, Y) / ( \sigma ( X) \sigma ( Y)) $,  
 +
where $  \sigma ( X) $
 +
is the variance of $  X $,
 +
$  \mathop{\rm var} ( X) = \mathop{\rm cov} ( X, X) $(
 +
cf. [[Dispersion|Dispersion]]), and there is no special name for the mixed second-order moment $  E( XY) $.  
 +
Correspondly one has the terms covariance function and correlation function, also termed auto-correlation function and auto-covariance function, for the following quantities associated to a stochastic process:
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280186.png" /></td> </tr></table>
+
$$
 +
B( s, t)  = {\mathsf E} [( X( s) - {\mathsf E} X( s)) ( X( t)- {\mathsf E} X( t)) ]
 +
$$
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280187.png" /></td> </tr></table>
+
$$
 +
\rho ( s, t)  = \sigma ( X( s))  ^ {-} 1 \sigma ( X( t))  ^ {-} 1 B( s, t) ,
 +
$$
  
and for two stochastic processes <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280188.png" /> there are correspondingly the cross covariance function and cross correlation function
+
and for two stochastic processes $  X( t), Y( t) $
 +
there are correspondingly the cross covariance function and cross correlation function
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280189.png" /></td> </tr></table>
+
$$
 +
B _ {XY} ( s, t )  = \
 +
{\mathsf E} [( X( s) - {\mathsf E} X( s)) ( Y( t) - {\mathsf E} Y( t)) ] ,
 +
$$
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280190.png" /></td> </tr></table>
+
$$
 +
\rho _ {XY} ( s, t)  = \sigma ( X( s))  ^ {-} 1 \sigma ( Y( t))  ^ {-} 1 B _ {XY} ( s, t).
 +
$$
  
However, in application areas of probability theory a somewhat different terminology is also employed, for instance in oceanology, hydrology and electrical engineering (cf. {{Cite|J}}–{{Cite|So}}). E.g., one also finds the phrase "correlation function" for the quantity <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280191.png" /> (instead of covariance function). Also, one regularly finds the terminology "correlation function" for the quantity
+
However, in application areas of probability theory a somewhat different terminology is also employed, for instance in oceanology, hydrology and electrical engineering (cf. {{Cite|J}}–{{Cite|So}}). E.g., one also finds the phrase "correlation function" for the quantity $  B( s, t) $(
 +
instead of covariance function). Also, one regularly finds the terminology "correlation function" for the quantity
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280192.png" /></td> </tr></table>
+
$$
 +
\gamma ( s, t)  = {\mathsf E} [ X( s) X( t)] ,
 +
$$
  
 
which is also in agreement with the phrase "correlation function" (pair correlation function) as it is used in statistical mechanics, cf. e.g. {{Cite|T}}.
 
which is also in agreement with the phrase "correlation function" (pair correlation function) as it is used in statistical mechanics, cf. e.g. {{Cite|T}}.
  
If the process is stationary, the distinctions are minor. Thus, if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280193.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280194.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280195.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s087/s087280/s087280196.png" />.
+
If the process is stationary, the distinctions are minor. Thus, if $  m = {\mathsf E} X( s) $,
 +
$  \sigma = \sigma ( X( s)) $,
 +
$  B( s, t) = \gamma ( s, t) - m  ^ {2} $,
 +
$  \rho ( s, t) = \sigma  ^ {-} 2 B( s, t) $.
  
 
====References====
 
====References====

Revision as of 08:23, 6 June 2020


stochastic process, homogeneous in time

2020 Mathematics Subject Classification: Primary: 60G10 [MSN][ZBL]

A stochastic process $ X( t) $ whose statistical characteristics do not change in the course of time $ t $, i.e. are invariant relative to translations in time: $ t \rightarrow t + a $, $ X( t) \rightarrow X( t+ a) $, for any fixed value of $ a $( either a real number or an integer, depending on whether one is dealing with a stochastic process in continuous or in discrete time). The concept of a stationary stochastic process is widely used in applications of probability theory in various areas of natural science and technology, since these processes accurately describe many real phenomena accompanied by unordered fluctuations. For example, the pulsations of the force of a current or the voltage in an electrical chain (electrical "noise" ) can be considered as stationary stochastic processes if the chain is in a stationary system; the pulsations of velocity or pressure at a point of a turbulent flow are stationary stochastic processes if the flow is stationary, etc.

In the mathematical theory of stationary stochastic processes, an important role is played by the moments of the probability distribution of the process $ X( t) $, and especially by the moments of the first two orders — the mean value $ {\mathsf E} X( t) = m $, and its covariance function $ {\mathsf E} [( X( t + \tau ) - {\mathsf E} X( t + \tau )) ( X( t) - EX( t)) ] $, or, equivalently, the correlation function $ E X( t+ \tau ) X( t) = B( \tau ) $. In much of the research into the theory of stationary stochastic processes, the properties that are completely defined by the characteristics $ m $ and $ B( \tau ) $ alone are studied (the so-called correlation theory or theory of second-order stationary stochastic processes). Accordingly, the stochastic processes $ X( t) $ for which $ {\mathsf E} X( t) $ and $ {\mathsf E} X( t + \tau ) X( t) $ do not depend on $ t $ are often separated into a special class and are called stationary stochastic processes in the wide sense. The more special stochastic processes, none of whose characteristics change with time (so that the distribution function $ F _ {t _ {1} \dots t _ {n} } ( x _ {1} \dots x _ {n} ) $ of an $ n $- dimensional random variable $ \{ X( t _ {1} ) \dots X( t _ {n} ) \} $ depends here, for any $ n $, only on the $ n- 1 $ differences $ t _ {2} - t _ {1} \dots t _ {n} - t _ {1} $) are called stationary stochastic processes in the strict sense. Accordingly, the theory of stationary stochastic processes is divided into the theory of stationary stochastic processes in the strict sense and in the wide sense, with a different mathematical apparatus used in each.

In the strict sense, the theory can be stated outside the framework of probability theory as the theory of one-parameter groups of transformations of a measure space that preserve the measure; this theory is very close to the general theory of dynamical systems (cf. Dynamical system) and to ergodic theory. The most important general theorem of the theory of stationary stochastic processes in the strict sense is the Birkhoff–Khinchin ergodic theorem, according to which for any stationary stochastic process $ X( t) $ in the strict sense having a mathematical expectation (i.e. $ {\mathsf E} | X( t) | < \infty $), the limit

$$ \tag{1 } \lim\limits _ {T- S \rightarrow \infty } \frac{1}{T-} S \int\limits _ { S } ^ { T } X( t) dt = \widehat{X} , $$

or

$$ \tag{1a } \lim\limits _ {T- S \rightarrow \infty } \frac{1}{T-} S \sum _ { t= } S+ 1 ^ { T } X( t) = \widehat{X} , $$

exists with probability 1 (formula (1) relates to processes in continuous time, while (1a) relates to processes in discrete time). There is a result of E.E. Slutskii [Sl], related to stationary stochastic processes in the wide sense, which states that the limit (1) or (1a) exists in mean square. This limit coincides with $ {\mathsf E} X( t) $ if and only if

$$ \tag{2 } \lim\limits _ {T \rightarrow \infty } T ^ {-} 1 \int\limits _ { 0 } ^ { T } b( \tau ) d \tau = 0 , $$

or

$$ \tag{2a } \lim\limits _ {T \rightarrow \infty } \ T ^ {-} 1 \sum _ {\tau = 0 } ^ { T- } 1 b( \tau ) = 0, $$

where

$$ b( \tau ) = B( \tau ) - m ^ {2} = \ {\mathsf E} [ X( t+ \tau ) - m][ X( t) - m] $$

(the von Neumann ( $ L _ {2} $-) ergodic theorem). These conditions are satisfied, in particular, when $ b( \tau ) \rightarrow 0 $ as $ \tau \rightarrow \infty $. The Birkhoff–Khinchin theorem can be applied to various stationary stochastic processes in the strict sense of the form

$$ Y _ \Phi ( s) = \Phi [ X( t+ s)], $$

where $ \Phi [ X( t)] $ is an arbitrary functional of the stationary stochastic process $ X( t) $, and is a random variable which has a mathematical expectation; if for all such stationary stochastic processes $ Y _ \Phi ( s) $ the corresponding limit $ \widehat{Y} _ \Phi $ coincides with $ {\mathsf E} Y _ \Phi $, then $ X( t) $ is called a metrically transitive stationary stochastic process. For stationary Gaussian stochastic processes $ X( t) $, the condition of being stationary in the strict sense coincides with the condition of being stationary in the wide sense; metric transitivity will occur if and only if the spectral function $ F( \lambda ) $ of $ X( t) $ is a continuous function of $ \lambda $( see, for example, [R], [CL]). There are, in general, no simple necessary and sufficient conditions for the metric transitivity of a stationary stochastic process $ X( t) $.

Apart from the above result relating to metric transitivity, there are also other results specifically for stationary Gaussian stochastic processes. For these processes, detailed studies have been made of the question of the local properties of the realizations of $ X( t) $( i.e. of individual observed values), and of the statistical properties of the sequence of zeros or maxima of the realizations of $ X( t) $, and the points of intersection of it with a given level (see, for example, [CL]). A typical example of the results related to intersections with a level is the statement that, given broad regularity conditions, the set of points of intersection of a high level $ x= u $ with the stationary Gaussian stochastic process $ X( t) $ in a certain special time scale (dependent on $ u $ and tending rapidly to infinity when $ u \rightarrow \infty $) converges to a Poisson flow of events of unit intensity when $ u \rightarrow \infty $( see [CL]).

When studying stationary stochastic processes in the wide sense, the Hilbert space $ H _ {X} $ of linear combinations of values of the process $ X( t) $ and the mean-square limits of sequences of such linear combinations are examined, and a scalar product is defined in it by the formula $ ( Y _ {1} , Y _ {2} ) = {\mathsf E} Y _ {1} Y _ {2} $. In this case, the transformation $ X( t) \rightarrow X( t+ a) $, where $ a $ is a fixed number, will generate a linear unitary operator $ U _ {a} $ mapping the space $ H _ {X} $ onto itself; the family of operators $ U _ {a} $ clearly satisfies the condition $ U _ {a} U _ {b} = U _ {a+} b $, while the values $ X( t) = U _ {t} X( 0) $ form a set of points (a curve if the time $ t $ is continuous, and a countable sequence of points if the time is discrete), mapped onto itself by all operators $ U _ {a} $. Accordingly, the theory of stationary stochastic processes in the wide sense can be reformulated in terms of functional analysis as the study of the sets of points $ X( t) = U _ {t} X( 0) $ of the Hilbert space $ H _ {X} $, where $ U _ {t} $ is a family of linear unitary operators such that $ U _ {a} U _ {b} = U _ {a+} b $( cf. also Semi-group of operators).

Spectral considerations, based on the expansion of the stochastic process $ X( t) $ and its correlation function $ B( \tau ) $ into a Fourier–Stieltjes integral, are central to the theory of stationary stochastic processes in the wide sense. By Khinchin's theorem [Kh] (which is a simple consequence of Bochner's analytic theorem on the general form of a positive-definite function), the correlation function $ B( \tau ) $ of a stationary stochastic process in continuous time can always be represented in the form

$$ \tag{3 } B( \tau ) = \int\limits _ \Lambda e ^ {i \tau \lambda } dF( \lambda ), $$

where $ F( \lambda ) $ is a bounded monotone non-decreasing function of $ \lambda $, while $ \Lambda = (- \infty , \infty ) $; the Herglotz theorem on the general form of positive-definite sequences similarly shows that the same representation, but with $ \Lambda = [- \pi , \pi ] $, also holds for the correlation function of a stationary stochastic process in discrete time. If the correlation function $ B( \tau ) $ decreases sufficiently rapidly as $ | \tau | \rightarrow \infty $( as is most often the case in applications under the condition that by $ X( t) $ one understands the difference $ X( t)- m $, i.e. it is considered that $ {\mathsf E} X( t) = 0 $), then the integral at the right-hand side of (3) becomes an ordinary Fourier integral

$$ \tag{4 } B( \tau ) = \int\limits _ \Lambda e ^ {i \tau \lambda } f( \lambda ) d \lambda , $$

where $ f( \lambda ) = F ^ { \prime } ( \lambda ) $ is a non-negative function. The function $ F( \lambda ) $ is called the spectral function of $ X( t) $, while the function $ f( \lambda ) $( in cases where the equality (4) holds) is called its spectral density. Starting from the Khinchin formula (3) (or from the definition of the process $ X( t) $ in the form of the set of points $ X( t) = U _ {t} X( 0) $ in the Hilbert space $ H _ {X} $, and Stone's theorem on the spectral representation of one-parameter groups of unitary operators in a Hilbert space), it can also be demonstrated that the process $ X( t) $ itself permits a spectral decomposition in the form

$$ \tag{5 } X( t) = \int\limits _ \Lambda e ^ {it \lambda } dZ( \lambda ), $$

where $ Z( \lambda ) $ is a random function with uncorrelated increments (i.e. $ {\mathsf E} dZ( \lambda _ {1} ) dZ( \lambda _ {2} ) = 0 $ when $ \lambda _ {1} \neq \lambda _ {2} $) which satisfies the condition $ {\mathsf E} | dZ( \lambda ) | ^ {2} = dF( \lambda ) $, while the integral at the right-hand side is understood to be the mean-square limit of the corresponding sequence of integral sums. The decomposition (5) provides grounds for considering any stationary stochastic process in the wide sense as a superposition of a set of non-correlated harmonic oscillations of different frequencies with random amplitudes and phases; the spectral function $ F( \lambda ) $ and the spectral density $ f( \lambda ) $ define the distribution of the average energy (or, more accurately, of the power) of the harmonic oscillations with frequency spectrum $ \lambda $ that constitute $ X( t) $( as a result of which the function $ f( \lambda ) $ in applied research is often called the energy spectrum, or power spectrum, of $ X( t) $).

The spectral decomposition of the correlation function $ B( \tau ) $, defined by formula (3), demonstrates that the mapping $ X( t) \rightarrow e ^ {it \lambda } $, which maps elements $ X( t) $ of the Hilbert space $ H _ {X} $ to elements $ e ^ {it \lambda } $ of the Hilbert space $ L _ {2} ( dF ) $ of complex-valued functions on the set $ \Lambda $ with a modulus, square-integrable with respect to $ dF( \lambda ) $, is an isometric mapping of $ H $ into $ L _ {2} ( dF ) $. This mapping can be extended to an isometric linear mapping $ M $ of the whole space $ H _ {X} $ onto the space $ L _ {2} ( dF ) $, a fact that allows one to reformulate many problems in the theory of stationary stochastic processes in the wide sense as problems in function theory.

A significant part of the theory of stationary stochastic processes in the wide sense is devoted to methods of solving linear approximation problems for such processes, i.e. methods of locating a linear combination of any "known" values of $ X( t) $ that best approximates (in the sense of the minimum least-square error) a certain "unknown" value of the same process or any "unknown" random variable $ Y $. In particular, the problem of optimal linear extrapolation of $ X( t) $ consists of finding the best approximation $ X ^ {*} ( s) $ of the value $ X( s) $, $ s > 0 $, that linearly depends on the "past values" of $ X( t) $ with $ t \leq 0 $; the problem of optimal linear interpolation consists of finding the best approximation for $ X( s) $ that linearly depends on the values of $ X( t) $, where $ t $ runs through all values that do not belong to a specific interval of the time axis (to which $ s $ does belong); the problem of optimal linear filtering can be formulated as the problem of finding the best approximation $ Y ^ {*} $ for a certain random variable $ Y $( which is usually the value for some $ t= s $ of a stationary stochastic process $ Y( t) $, correlated with $ X( t) $, whereby $ Y( t) $ most often plays the part of a "signal" , while $ X( t) = Y( t) + N( t) $ is the sum of the "signal" and a "noise" $ N( t) $ that interferes with it, and the sum is known from the observations) that linearly depends on the values of $ X( t) $ when $ t \leq 0 $( see Stochastic processes, prediction of; Stochastic processes, filtering of; Stochastic processes, interpolation of).

All these problems reduce geometrically to the problem of projecting a point of the Hilbert space $ H _ {X} $( or of its extension) orthogonally onto a given subspace of this space. Relying on this geometric interpretation and on the isomorphism of the spaces $ H _ {X} $ and $ L _ {2} ( dF ) $, A.N. Kolmogorov has deduced general formulas that make it possible to determine the mean-square error of optimal linear extrapolation or interpolation, corresponding to the case where the value of $ X( t) $ is unknown only when $ t= s $, by means of the spectral function $ F( \lambda ) $ of the stationary stochastic process $ X( t) $ in discrete time $ t $( see [R], [Ko][D]). When used for the extrapolation problem, the same results were obtained for processes $ X( t) $ in continuous time by M.G. Krein and K. Karhunen. N. Wiener [W] demonstrated that the search for the best approximation $ X ^ {*} ( s) $ or $ Y ^ {*} = Y ^ {*} ( s) $ in the case of problems of optimal linear extrapolation and filtering can be reduced to the solution of a certain integral equation of Wiener–Hopf type, or (when $ t $ is discrete) of the discrete analogue of such an equation, which makes it possible to use the factorization method (see Wiener–Hopf equation; Wiener–Hopf method). Problems of optimal linear extrapolation or filtering of a stationary stochastic process $ X( t) $ in continuous time in the case where not all its past values for $ t \leq 0 $ are known but only its values on a finite interval $ - T \leq t \leq 0 $, as well as the problem of optimal linear interpolation of such an $ X( t) $, can be reduced to certain problems of establishing a special form of differential equation (a "generalized string equation" ) by means of its spectrum (see [Kr], [DM]).

The above approaches to the solution of problems of optimal linear extrapolation, interpolation and filtering provide sufficiently-simple explicit formulas for the required best approximation $ X ^ {*} ( s) $ or $ Y ^ {*} $ that can be successfully used in practice only in certain exceptional cases. One important case in which such explicit formulas do exist is the case of a stationary stochastic process $ X( t) $ with rational spectral density $ f( \lambda ) $ relative to the $ e ^ {i \lambda } $( if $ t $ is discrete) or relative to $ \lambda $( if $ t $ is continuous), which was studied in detail by Wiener [W] (for applications to problems of extrapolation and filtering by values where $ t \leq 0 $). It was subsequently demonstrated that for such stationary stochastic processes with rational spectral density there is also an explicit solution of the problems of linear interpolation, extrapolation and filtering by means of data on a finite interval $ - T \leq t \leq 0 $( see, for example, [R], [Y]). The simplicity of processes with rational spectral density can be explained by the fact that such stationary stochastic processes (and practically only they) are a one-dimensional component of a multi-dimensional stationary Markov process (see [D2]).

The concept of a stationary stochastic process permits a whole series of generalizations. One of these is the concept of a generalized stationary stochastic process. This is a generalized stochastic process (cf. Stochastic process, generalized) $ X( \phi ) $( i.e. a random linear functional, defined on the space $ D $ of infinitely-differentiable functions $ \phi ( t) $ with compact support), such that either the distribution function of the random vector $ \{ X( V _ {a} \phi _ {1} ) \dots X( V _ {a} \phi _ {n} ) \} $, where $ V _ {a} \phi ( t) = \phi ( t+ a) $ for any positive integer $ n $, real number $ a $ and $ \phi _ {1} \dots \phi _ {2} \in D $, coincides with the probability distribution of the vector $ \{ X( \phi _ {1} ) \dots X( \phi _ {n} ) \} $( a generalized stationary stochastic process in the strict sense), or else

$$ {\mathsf E} X( \phi ) = {\mathsf E} X( V _ {a} \phi ), $$

$$ {\mathsf E} X( V _ {a} \phi _ {1} ) \overline{ {X( V _ {a} \phi _ {2} ) }}\; = {\mathsf E} X( \phi _ {1} ) \overline{ {X( \phi _ {2} ) }}\; , $$

for all $ a $( a generalized stationary stochastic process in the wide sense). A generalized stationary stochastic process in the wide sense $ X( \phi ) $ and its correlation functional $ B( \phi _ {1} , \phi _ {2} ) = {\mathsf E} X( \phi _ {1} ) \overline{ {X( \phi _ {2} ) }}\; $( or covariance functional $ {\mathsf E} [ ( X( \phi _ {1} ) - {\mathsf E} X( \phi _ {1} )) \overline{ {( X( \phi _ {2} ) - {\mathsf E} X( \phi _ {2} )) }}\; ] $) permit a spectral decomposition related to (2) and (5) (see Spectral decomposition of a random function). Other frequently-used generalizations of the concept of a stationary stochastic process are the concepts of a stochastic process with stationary increments of a certain order and of a homogeneous random field (cf. Random field, homogeneous).

References

[Sl] E.E. Slutskii, Selected works , Moscow (1980) pp. 252–268 (In Russian)
[R] Yu.A. Rozanov, "Stationary random processes" , Holden-Day (1967) (Translated from Russian) MR0214134 Zbl 0152.16302
[CL] H. Cramér, M.R. Leadbetter, "Stationary and related stochastic processes" , Wiley (1967) MR0217860 Zbl 0162.21102
[Kh] A.Ya. Khinchin, Uspekhi Mat. Nauk : 5 (1938) pp. 42–51
[Ko] A.N. Kolmogorov, "Interpolation and extrapolation of stationary stochastic series" Izv. Akad. Nauk SSSR Ser. Mat. , 5 : 1 (1941) pp. 3–14 (In Russian) (German abstract)
[D] J.L. Doob, "Stochastic processes" , Wiley (1953) MR1570654 MR0058896 Zbl 0053.26802
[GS] I.I. Gihman, A.V. Skorohod, "The theory of stochastic processes" , 1 , Springer (1971) (Translated from Russian) MR0636254 MR0651015 MR0375463 MR0346882 Zbl 0531.60002 Zbl 0531.60001 Zbl 0404.60061 Zbl 0305.60027 Zbl 0291.60019
[W] N. Wiener, "Extrapolation, interpolation and smoothing of stationary time series" , M.I.T. (1949) MR0031213 Zbl 0036.09705
[Kr] M.G. Krein, "On a basic approximation problem of the theory of extrapolation and filtering of stationary stochastic processes" Dokl. Akad. Nauk SSSR , 94 : 1 (1954) pp. 13–16 (In Russian)
[DM] H. Dym, H.P. McKean, "Gaussian processes, function theory, and the inverse spectral problem" , Acad. Press (1976) MR0448523 Zbl 0327.60029
[Y] A.M. Yaglom, "Extrapolation, interpolation and filtering of stationary stochastic processes with rational spectral density" Trudy Moskov. Mat. Obshch. , 4 (1955) pp. 333–374 (In Russian)
[D2] J.L. Doob, "The elementary Gaussian processes" Ann. Math. Stat. , 15 (1944) pp. 229–282 MR0010931 Zbl 0060.28907

Comments

In the English language literature one says ergodic stationary process rather than metrically-transitive stationary process. See also Gaussian process.

The terminology concerning the phrases "correlation function" and "covariance function" is not yet completely standardized. In probability theory the following terminology seems just about universally adopted. Given two random variables $ X, Y $ their covariance is $ \mathop{\rm cov} ( X, Y) = {\mathsf E} [( X- {\mathsf E} X )( Y- {\mathsf E} Y )] $, their correlation coefficient is $ \rho ( X, Y) = \mathop{\rm cov} ( X, Y) / ( \sigma ( X) \sigma ( Y)) $, where $ \sigma ( X) $ is the variance of $ X $, $ \mathop{\rm var} ( X) = \mathop{\rm cov} ( X, X) $( cf. Dispersion), and there is no special name for the mixed second-order moment $ E( XY) $. Correspondly one has the terms covariance function and correlation function, also termed auto-correlation function and auto-covariance function, for the following quantities associated to a stochastic process:

$$ B( s, t) = {\mathsf E} [( X( s) - {\mathsf E} X( s)) ( X( t)- {\mathsf E} X( t)) ] $$

$$ \rho ( s, t) = \sigma ( X( s)) ^ {-} 1 \sigma ( X( t)) ^ {-} 1 B( s, t) , $$

and for two stochastic processes $ X( t), Y( t) $ there are correspondingly the cross covariance function and cross correlation function

$$ B _ {XY} ( s, t ) = \ {\mathsf E} [( X( s) - {\mathsf E} X( s)) ( Y( t) - {\mathsf E} Y( t)) ] , $$

$$ \rho _ {XY} ( s, t) = \sigma ( X( s)) ^ {-} 1 \sigma ( Y( t)) ^ {-} 1 B _ {XY} ( s, t). $$

However, in application areas of probability theory a somewhat different terminology is also employed, for instance in oceanology, hydrology and electrical engineering (cf. [J][So]). E.g., one also finds the phrase "correlation function" for the quantity $ B( s, t) $( instead of covariance function). Also, one regularly finds the terminology "correlation function" for the quantity

$$ \gamma ( s, t) = {\mathsf E} [ X( s) X( t)] , $$

which is also in agreement with the phrase "correlation function" (pair correlation function) as it is used in statistical mechanics, cf. e.g. [T].

If the process is stationary, the distinctions are minor. Thus, if $ m = {\mathsf E} X( s) $, $ \sigma = \sigma ( X( s)) $, $ B( s, t) = \gamma ( s, t) - m ^ {2} $, $ \rho ( s, t) = \sigma ^ {-} 2 B( s, t) $.

References

[J] A.H. Jazwinski, "Stochastic processes and filtering theory" , Acad. Press (1970) pp. 53–54 Zbl 0203.50101
[SSN] Y. Sawaragi, Y. Sunahara, T. Nakamizo, "Statistical decision theory in adaptive control systems" , Acad. Press (1967) Zbl 0189.47003
[MO] A.S. Monin, R.V. Ozmidov, "Turbulence in the ocean" , Reidel (1985) (Translated from Russian)
[So] K. Sobczyk, "Stochastic differential equations. With applications to physics and engineering" , Kluwer (1991) pp. 23 MR1135326 Zbl 0762.60050
[T] C.J. Thompson, "Mathematical statistical mechanics" , Princeton Univ. Press (1972) MR0469020 Zbl 0244.60082
How to Cite This Entry:
Stationary stochastic process. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Stationary_stochastic_process&oldid=26947
This article was adapted from an original article by A.M. Yaglom (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article