View source for Stochastic processes, prediction of

''extrapolation of stochastic processes''

The problem of estimating the values of a [[Stochastic process|stochastic process]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902801.png" /> in the future <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902802.png" /> from its observed values up to the current moment of time <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902803.png" />. Usually one has in mind the extrapolation estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902804.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902805.png" />, for which the mean-square error <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902806.png" /> is minimal over all estimators based on the past values of the process up to the moment <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902807.png" /> (the prediction is called linear if one restricts consideration to linear estimators).

One of the problems posed and solved was that of linear prediction of a stationary sequence. This problem is analogous to the following: In the space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902808.png" /> of square-integrable functions on the interval <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s0902809.png" /> one has to find the projection of the function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028010.png" /> onto the subspace generated by the functions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028011.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028012.png" />. This problem has been greatly generalized in the theory of stationary stochastic processes (cf. [[Stationary stochastic process|Stationary stochastic process]]). One application is the problem of predicting stochastic processes arising from the system

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028013.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028014.png" /> is a linear differential operator of order <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028015.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028016.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028017.png" />, is a [[White noise|white noise]] process. The optimal prediction <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028018.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028019.png" />, given the values of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028020.png" /> at the times <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028021.png" /> and the initial values <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028022.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028023.png" />, independent of white noise, is obtained by solving the corresponding equation

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028024.png" /></td> </tr></table>

with initial conditions

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/s/s090/s090280/s09028025.png" /></td> </tr></table>

For systems of stochastic differential equations, the problem of predicting some components given the values of other observed components reduces to the extrapolation of the corresponding stochastic equations.

For references see [[Stochastic processes, interpolation of|Stochastic processes, interpolation of]].



====Comments====
Study of the linear prediction problem was initiated by A.N. Kolmogorov in Russia [[#References|[a7]]], [[#References|[a8]]], and in the West by N. Wiener [[#References|[a6]]], who also, with P. Masani, treated the multivariate case [[#References|[a11]]], [[#References|[a12]]]. They considered prediction based on the entire past; prediction based on a finite segment of the past is more subtle, see [[#References|[a9]]], [[#References|[a10]]]. Linear prediction for non-stationary processes is covered in [[#References|[a13]]]. The linear theory may also be found in [[#References|[a2]]]. The approach of R.E. Kalman [[#References|[a3]]], [[#References|[a4]]] has produced practically usable prediction algorithms, based on the Kalman filter and on state space realizations of the underlying processes. A generalization of the prediction problem to diffusion processes is presented in [[#References|[a5]]]. For applications of prediction algorithms in the applied sciences see [[#References|[a1]]].

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  G.E.P. Box,   G.M. Jenkins,   "Time series analysis. Forecasting and control" , Holden-Day  (1976)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  E.J. Hannan,   "Multiple time series" , Wiley  (1970)</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  R.E. Kalman,   "A new approach to linear filtering and prediction problems"  ''J. Basic Eng., Trans. ASME, Series D'' , '''82''' :  1  (March 1960)  pp. 35–45</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  R.E. Kalman,   R.S. Bucy,   "New results in linear filtering and prediction theory"  ''J. Basic Eng., Trans. ASME, Series D'' , '''83'''  (1961)  pp. 95–108</TD></TR><TR><TD valign="top">[a5]</TD> <TD valign="top">  E. Pardoux,   "Equations du filtrage nonlinéaire, de la prédiction et du lissage"  ''Stochastics'' , '''6'''  (1982)  pp. 193–231</TD></TR><TR><TD valign="top">[a6]</TD> <TD valign="top">  N. Wiener,   "Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications" , M.I.T.  (1949)</TD></TR><TR><TD valign="top">[a7]</TD> <TD valign="top">  A.N. Kolmogorov,   "Stationary sequences in Hilbert space"  ''Byull. Moskov. Gos. Univ. Sect. A'' , '''2''' :  6  (1941)  pp. 1–40  (In Russian)</TD></TR><TR><TD valign="top">[a8]</TD> <TD valign="top">  A.N. Kolmogorov,   "Interpolation und Extrapolation von stationären zufalligen Folgen"  ''Izv. Akad. Nauk. SSSR'' , '''5'''  (1941)  pp. 3–14</TD></TR><TR><TD valign="top">[a9]</TD> <TD valign="top">  M.G. Krein,   "On a fundamental approximation problem in the theory of extrapolation and filtration of stationary random processes"  ''Amer. Math. Soc. Sel. Transl. Math. Statist.'' , '''4'''  (1964)  pp. 127–131  ''Dokl. Akad. Nauk. SSSR'' , '''94'''  (1954)  pp. 13–16</TD></TR><TR><TD valign="top">[a10]</TD> <TD valign="top">  H. Dym,   H.P. McKean,   "Gaussian processes, function theory and the inverse spectral problem" , Acad. Press  (1976)</TD></TR><TR><TD valign="top">[a11]</TD> <TD valign="top">  N. Wiener,   P. Masani,   "The prediction theory of multivariate stochastic processes I"  ''Acta Math.'' , '''98'''  (1957)  pp. 111–150</TD></TR><TR><TD valign="top">[a12]</TD> <TD valign="top">  N. Wiener,   P. Masani,   "The prediction theory of multivariate stochastic processes II"  ''Acta Math.'' , '''99'''  (1958)  pp. 93–137</TD></TR><TR><TD valign="top">[a13]</TD> <TD valign="top">  J. Rissanen,   L. Barbosa,   "A factorization problem and the problem of predicting non-stationary vector-valued random processes"  ''Z. Wahrscheinlichkeitstheorie verw. Geb.'' , '''12'''  (1969)  pp. 255–266</TD></TR></table>