View source for Wold decomposition

A decomposition introduced by H. Wold in 1938 (see [[#References|[a7]]]); see also [[#References|[a5]]], [[#References|[a8]]]. Standard references include [[#References|[a6]]], [[#References|[a3]]].

The Wold decomposition of a (weakly) [[Stationary stochastic process|stationary stochastic process]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301701.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301702.png" />, provides interesting insights in the structure of such processes and, in particular, is an important tool for forecasting (from an infinite past).

The main result can be summarized as:

1) Every (weakly) stationary process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301703.png" /> can uniquely be decomposed as

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301704.png" /></td> </tr></table>

where the stationary processes <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301705.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301706.png" /> are obtained by causal linear transformations of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301707.png" /> (where  "causal"  means that, e.g. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301708.png" />, only depends on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w1301709.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017010.png" />), <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017011.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017012.png" /> are mutually uncorrelated, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017013.png" /> is linearly regular (i.e. the best linear least squares predictors converge to zero, if the forecasting horizon tends to infinity) and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017014.png" /> is linearly singular (i.e. the prediction errors for the best linear least squares predictors are zero).

2) Every linearly regular process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017015.png" /> can be represented as

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017016.png" /></td> <td valign="top" style="width:5%;text-align:right;">(a1)</td></tr></table>

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017017.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017018.png" /> is [[White noise|white noise]] (i.e. <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017019.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017020.png" />) and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017021.png" /> is obtained by a causal linear transformation of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017022.png" />.

The construction behind the Wold decomposition in the [[Hilbert space|Hilbert space]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017023.png" /> spanned by the one-dimensional process variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017024.png" /> is as follows: If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017025.png" /> denotes the subspace spanned by <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017026.png" />, then <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017027.png" /> is obtained from projecting <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017028.png" /> on the space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017029.png" />, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017030.png" /> is obtained as the perpendicular by projecting <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017031.png" /> on the space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017032.png" /> spanned by <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017033.png" />. Thus <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017034.png" /> is the innovation and the one-step-ahead prediction error for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017035.png" /> as well as for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017036.png" />.

The implications of the above-mentioned results for (linear least squares) prediction are straightforward: Since <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017037.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017038.png" /> are orthogonal and since <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017039.png" /> is the direct sum of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017040.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017041.png" />, the prediction problem can be solved for the linearly regular and the linearly singular part separately, and for a linearly regular process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017042.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017043.png" /> implies that the best linear least squares <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017044.png" />-step ahead predictor for <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017045.png" /> is given by

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017046.png" /></td> </tr></table>

and thus the prediction error is

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017047.png" /></td> </tr></table>

Thus, when the representation (a1) is available, the prediction problem for a linearly regular process can be solved.

The next problem is to obtain (a1) from the second moments of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017048.png" /> (cf. also [[Moment|Moment]]). The problem of determining the coefficients <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017049.png" /> of the Wold representation (a1) (or, equivalently, of determining the corresponding transfer function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017050.png" />) from the [[Spectral density|spectral density]]

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017051.png" /></td> <td valign="top" style="width:5%;text-align:right;">(a2)</td></tr></table>

(where the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017052.png" /> denotes the conjugate transpose) of a linearly regular process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017053.png" />, is called the spectral factorization problem. The following result holds:

3) A stationary process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017054.png" /> with a spectral density <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017055.png" />, which is non-singular <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017056.png" />-a.e., is linearly regular if and only if

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017057.png" /></td> </tr></table>

In this case the factorization <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017058.png" /> in (a2) corresponding to the Wold representation (a1) satisfies the relation

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017059.png" /></td> </tr></table>

The most important special case is that of rational spectral densities; for such one has (see e.g. [[#References|[a4]]]):

4) Any rational and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017060.png" />-a.e. non-singular spectral density <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017061.png" /> can be uniquely factorized, such that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017062.png" /> (the extension of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017063.png" /> to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017064.png" />) is rational, analytic within a circle containing the closed unit disc, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017065.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017066.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017067.png" /> (and thus corresponds to the Wold representation (a1)), and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017068.png" />. Then (a1) is the solution of a stable and miniphase ARMA or a (linear) finite-dimensional state space system.

Evidently, the Wold representation (a1) relates stationary processes to linear systems with white noise inputs. Actually, Wold introduced (a1) as a joint representation for AR and MA systems (cf. also [[Mixed autoregressive moving-average process|Mixed autoregressive moving-average process]]).

The Wold representation is used, e.g., for the construction of the state space of a linearly regular process and the construction of state space representations, see [[#References|[a1]]], [[#References|[a4]]]. As mentioned already, the case of rational transfer functions corresponding to stable and miniphase ARMA or (finite-dimensional) state space systems is by far the most important one. In this case there is a wide class of identification procedures available, which also give estimates of the coefficients <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017069.png" /> from finite data <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017070.png" /> (see e.g. [[#References|[a4]]]).

Another case is that of stationary long memory processes (see e.g. [[#References|[a2]]]). In this case, in (a1), <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017071.png" />, so that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/w/w130/w130170/w13017072.png" /> is infinity at frequency zero, which causes the long memory effect. Models of this kind, in particular so-called ARFIMA models, have attracted considerable attention in modern econometrics.

====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  H. Akaike,   "Stochastic theory of minimal realizations"  ''IEEE Trans. Autom. Control'' , '''AC-19'''  (1974)  pp. 667–674</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  C.W.J. Granger,   R. Joyeux,   "An introduction to long memory time series models and fractional differencing"  ''J. Time Ser. Anal.'' , '''1'''  (1980)  pp. 15–39</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  E.J. Hannan,   "Multiple time series" , Wiley  (1970)</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  E.J. Hannan,   M. Deistler,   "The statistical theory of linear systems" , Wiley  (1988)</TD></TR><TR><TD valign="top">[a5]</TD> <TD valign="top">  A.N. Kolmogorov,   "Stationary sequences in Hilbert space"  ''Bull. Moscow State Univ.'' , '''2''' :  6  (1941)  pp. 1–40</TD></TR><TR><TD valign="top">[a6]</TD> <TD valign="top">  Y.A. Rozanov,   "Stationary random processes" , Holden Day  (1967)</TD></TR><TR><TD valign="top">[a7]</TD> <TD valign="top">  H. Wold,   "Study in the analysis of stationary time series" , Almqvist and Wiksell  (1954)  (Edition: Second)</TD></TR><TR><TD valign="top">[a8]</TD> <TD valign="top">  V.N. Zasukhin,   "On the theory of multidimensional stationary processes"  ''Dokl. Akad. Nauk SSSR'' , '''33'''  (1941)  pp. 435–437</TD></TR></table>