Spectral decomposition of a random function

spectral representation of a random function

A representation of a random function (in particular, of a stochastic process) by a series or integral with respect to some special system of functions, such that the coefficients in this expansion are pairwise uncorrelated random variables. A wide class of spectral representations of complex-valued random functions $ X ( t) $, $ t \in T $, with zero mean value (that is, such that $ {\mathsf E} X ( t) = 0 $) can be written in the form

$$ \tag{1 } X ( t) = \int\limits _ \Lambda \phi ( t ; \lambda ) Z ( d \lambda ) , $$

where $ \Lambda $ is some set with a given system of "measurable subsets" (that is, a measurable space); $ \phi ( t ; \lambda ) $, $ t \in T $, $ \lambda \in \Lambda $, is a system of complex-valued functions of $ t \in T $, depending on a parameter $ \lambda \in \Lambda $; $ Z ( d \lambda ) $ is an orthogonal random measure on $ \Lambda $( with uncorrelated values, so that $ {\mathsf E} Z ( \Delta _ {1} ) \overline{ {Z ( \Delta _ {2} ) }}\; = 0 $ for any two disjoint measurable subsets $ \Delta _ {1} $ and $ \Delta _ {2} $); and the integral on the right-hand side can be defined either as the mean-square limit of the corresponding sequence of integral Cauchy sums (), or, more generally, as a "Lebesgue integral with respect to the measure Zdl" (see , for example). According to the general Karhunen spectral representation theorem, a spectral representation (1) exists for a random function $ X ( t) $ if and only if the corresponding correlation function $ B ( t , s ) = {\mathsf E} X ( t) \overline{ {X ( s) }}\; $ can be written in the form

$$ B ( t , s ) = \int\limits _ \Lambda \phi ( t ; \lambda ) {\phi ( s ; \lambda ) } bar F ( d \lambda ) , $$

where $ F ( d \lambda ) = {\mathsf E} | Z ( d \lambda ) | ^ {2} $ is a non-negative measure on $ \Lambda $.

The best known class of spectral representations of random functions are the representations of stationary stochastic processes $ X ( t) $ as Fourier–Stieltjes integrals,

$$ \tag{2 } X ( t) = \int\limits _ \Lambda e ^ {i t \lambda } d Z ( \lambda ) , $$

where $ Z ( \lambda ) $ is a random function of $ \lambda $ with uncorrelated increments and $ \Lambda $ is either the real line $ ( - \infty , \infty ) $, when the time $ t $ is continuous, or the interval $ [ - \pi , \pi ] $, when $ t $ is discrete (and takes integral values). The existence of such a spectral decomposition follows from the general theorem of Khinchin (or Wiener–Khinchin) on the integral representation of the correlation function $ B ( s) = {\mathsf E} X ( t + s ) X ( t) $( see Stationary stochastic process). This shows that any stationary stochastic process can be regarded as the superposition of mutually uncorrelated harmonic oscillations of various frequencies and with random phases and amplitudes. A spectral decomposition of similar form, but with $ n $- dimensional planar waves in place of harmonic oscillations, also exists for homogeneous random fields defined on a Euclidean $ n $- dimensional space $ \mathbf R ^ {n} $, or on the lattice $ \mathbf Z ^ {n} $ of integer points in $ \mathbf R ^ {n} $. In the case of a generalized stationary stochastic process, consider a linear functional $ X ( \phi ) $ on the space $ D $ of infinitely-differentiable functions $ \phi ( t) $ of compact support satisfying the conditions

$$ {\mathsf E} X ( V _ {a} \phi ) = {\mathsf E} X ( \phi ) , $$

$$ {\mathsf E} X ( V _ {a} \phi _ {1} ) {X ( V _ {a} \phi _ {2} ) } bar = {\mathsf E} X ( \phi _ {1} ) \overline{ {X ( \phi _ {2} ) }}\; $$

for all real $ a $, where $ V _ {a} \phi ( t) = \phi ( t + a ) $. The functional $ X ( \phi ) $ can be written in the form

$$ \tag{3 } X ( \phi ) = \int\limits _ {- \infty } ^ \infty \widetilde \phi ( \lambda ) d Z ( \lambda ) , $$

where

$$ \widetilde \phi ( \lambda ) = \ \int\limits _ {- \infty } ^ \infty e ^ {i \lambda t } \phi ( t) d t $$

is the Fourier transform of $ \phi ( t) $. Formula (3) follows from the fact that

$$ B ( \phi _ {1} , \phi _ {2} ) = \ {\mathsf E} X ( \phi _ {1} ) \overline{ {X ( \phi _ {2} ) }}\; $$

can be written in the form

$$ B ( \phi _ {1} , \phi _ {2} ) = \ \int\limits _ {- \infty } ^ \infty \widetilde \phi _ {1} ( \lambda ) \overline{ {\widetilde \phi _ {2} ( \lambda ) }}\; \ d F ( \lambda ) , $$

where the function $ F ( \lambda ) = {\mathsf E} | Z ( \lambda ) - Z ( - \infty ) | ^ {2} $ is a monotone non-decreasing spectral distribution function such that

$$ \int\limits _ {- \infty } ^ \infty ( 1 + \lambda ^ {2} ) ^ {-m} d F ( \lambda ) < \infty $$

for some non-negative integer $ m $( see ). If one takes the space of functions $ \phi ( t) $ to be some special space of entire analytic functions, then one arrives at a generalized stationary stochastic process $ X ( \phi ) $ with exponentially increasing spectral function $ F ( \lambda ) $( see ).

Spectral decompositions of special form also occur for homogeneous random fields on groups $ G $ and on homogeneous spaces $ S $. This is a consequence of Karhunen's spectral decomposition theorem together with certain well-known results on the general form of positive-definite functions (or kernels, which are functions in two variables) on the sets $ G $ and $ S $. In particular, for a homogeneous field $ X ( g) $ on an arbitrary locally compact Abelian group $ G $, the spectral representation of $ X ( g) $ has the form (1), where the role of the functions $ \phi ( t ; \lambda ) $ is played by the characters $ \chi ^ {( \lambda ) } ( g) $ of $ G $, and the domain $ \Lambda $ of integration is the corresponding character group $ \widehat{G} $( see, , , for example). Spectral representations of more complicated form also occur under fairly general conditions for homogeneous fields on non-commutative topological groups (see ). Finally, in the case of homogeneous fields on homogeneous spaces $ S = \{ s \} $, the spectral decomposition of a field $ X ( s) $ involves the spherical functions (spherical harmonics) in the space $ S $, and the corresponding spectral representation of the correlation function $ B ( s _ {1} , s _ {2} ) = {\mathsf E} X ( s _ {1} ) \overline{ {X ( s _ {2} ) }}\; $ involves the zonal spherical functions (see , ). In particular, a general homogeneous field $ X ( \theta , \phi ) $ on the sphere $ S _ {2} $ in the three-dimensional space $ \mathbf R ^ {3} $ admits a spectral representation of the form

$$ \tag{4 } X ( \theta , \phi ) = \sum_{l=0}^ \infty \ \sum _ {m = - l } ^ { l } Y _ {l,m} ( \theta , \phi ) Z _ {l,m} , $$

where

$$ Y _ {l,m} = e ^ {- i m \phi } P _ {l} ^ {m} ( \cos \theta ) $$

are ordinary spherical harmonics and the random variables $ Z _ {l,m} $ are such that $ {\mathsf E} Z _ {l,m} \overline{Z}\; _ {j,n} = \delta _ {lj} \delta _ {mn} f _ {l} $, where $ \delta _ {lj} $ is the Kronecker delta. Corresponding to formula (4) there is an expression for the correlation function of the form

$$ {\mathsf E} X ( \theta _ {1} , \phi _ {1} ) \overline{ {X ( \theta _ {2} , \phi _ {2} ) }}\; = B ( \theta _ {12} ) , $$

where $ \theta _ {12} $ is the angular distance between the points $ ( \theta _ {1} , \phi _ {1} ) $ and $ ( \theta _ {2} , \phi _ {2} ) $, and

$$ B ( \theta ) = \sum_{l=0}^ \infty \frac{2 l + 1 }{2} f _ {l} P _ {l} ( \cos \theta ) , $$

where the $ P _ {l} $ are the Legendre polynomials. Similarly, if $ X ( r , \phi ) $( where $ ( r , \phi ) $ are polar coordinates) is a homogeneous and isotropic field in the plane $ \mathbf R ^ {2} $( so that $ {\mathsf E} X ( r _ {1} , \phi _ {1} ) \overline{ {X ( r _ {2} , \phi _ {2} ) }}\; = B ( r _ {12} ) $, where $ r _ {12} $ is the Euclidean distance between the points $ ( r _ {1} , \phi _ {1} ) $ and $ ( r _ {2} , \phi _ {2} ) $), then the spectral representation of $ X ( r , \phi ) $ can be written in the form

$$ \tag{5 } X ( r , \phi ) = \sum _ {k = - \infty } ^ \infty e ^ {i k \phi } \int\limits _ { 0 } ^ \infty J _ {k} ( \lambda r ) d Z _ {k} ( \lambda ) , $$

where $ J _ {k} ( x) $ is the Bessel function of order $ k $( cf Bessel functions). Here the $ Z _ {k} ( \lambda ) $ are random functions with uncorrelated increments such that

$$ {\mathsf E} Z _ {k} ( \Delta _ {1} ) \overline{ {Z _ {m} ( \Delta _ {2} ) }}\; = \ \delta _ {km} F ( \Delta _ {1} \cap \Delta _ {2} ) , $$

where

$$ Z _ {k} ( \Delta ) = \int\limits _ \Delta d Z _ {k} ( \lambda ) $$

and $ F ( \Delta ) $ is a non-negative measure on the semi-axis $ [ 0 , \infty ) $. Corresponding to the spectral representation (5), there is the following expression for the correlation function $ B ( r) $:

$$ B ( r) = \int\limits _ { 0 } ^ \infty J _ {0} ( \lambda r ) d F ( \lambda ) . $$

For further examples of spectral representations of homogeneous fields see –.

Spectral representations of random functions exist not only for stationary stochastic processes and homogeneous fields. For example, if $ X ( t) $ is an arbitrary stochastic process on the interval $ a \leq t \leq b $ with correlation function

$$ B ( t , s ) = {\mathsf E} X ( t) \overline{ {X ( s) }}\; $$

which is continuous in both arguments, then by virtue of Mercer's theorem in the theory of integral equations (cf. Mercer theorem) and Karhunen's spectral representation theorem, $ X ( t) $ admits a spectral representation of the form

$$ \tag{6 } X ( t) = \sum_{k=1}^ \infty \frac{\phi _ {k} ( t) Z _ {k} }{\sqrt {\lambda _ {k} } } , $$

where the $ \phi _ {k} ( t) $ and the $ \lambda _ {k} $, $ k = 1 , 2 \dots $ are the eigenfunctions and eigenvalues of the integral operator on the function space with kernel $ B ( t , s ) $ and $ {\mathsf E} Z _ {k} \overline{Z}\; _ {j} = \delta _ {kj} $. The spectral representation (6) of a stochastic process $ X ( t) $, defined on a finite interval, is the continuous analogue of the decomposition of a random vector into its principal components, which is often used in multivariate statistical analysis. It was obtained independently by a number of scientists (see , , for example) and is most frequently called the Karhunen–Loève expansion. Spectral representations of the form (6) are widely used in many applications, in particular in the theory of automatic control, where (6) and some related representations are often called canonical representations of stochastic processes (see ), and in meteorology and geophysics, where the term "method of empirical orthogonal functionsmethod of empirical orthogonal functions" is usually used, since the eigenfunctions $ \phi _ {k} ( t) $ in practice must be approximately determined by empirical data (see , ).

A spectral representation of a random function $ X ( t) $, $ t \in T $, can sometimes also mean a general representation of the form

(without the requirement that $ Z( d \lambda ) $ is a random measure with uncorrelated values) into a certain standard (sufficiently simple) complete system of functions $ \phi ( t ; \lambda ) $. This is most common in the case of decompositions of stochastic processes $ X ( t) $ with continuous time into functions $ \phi ( t ; \lambda ) = e ^ {i t \lambda } $, so that

reduces to . It follows from

that, in general, $ B ( t , s ) = {\mathsf E} X ( t) \overline{ {X ( s) }}\; $ can be written in the form

$$ \tag{7 } B ( t , s ) = \int\limits _ {- \infty } ^ \infty \int\limits _ {- \infty } ^ \infty e ^ {i ( \lambda t - \mu s ) } F ( d \lambda \times d \mu ) , $$

where $ F ( d \lambda \times d \mu ) $ is the complex-valued measure on the $ ( \lambda , \mu ) $- plane defined by the relation

$$ F ( \Delta _ {1} , \Delta _ {2} ) = \ {\mathsf E} Z ( \Delta _ {1} ) \overline{ {Z ( \Delta _ {2} ) }}\; . $$

Conversely, it can be easily shown that the fact that $ B ( t , s ) $ can be written in the form (7) implies that there is a spectral representation

(see, for example, [2]). Stochastic processes admitting a spectral representation , where $ Z ( \lambda ) $ does not necessarily have uncorrelated increments, are called harmonizable stochastic processes. In this case, the complex measure $ F ( d \lambda \times d \mu ) $ is called the spectral measure of $ X ( t) $, and the set of points of the $ ( \lambda , \mu ) $- plane not having a neighbourhood of spectral measure zero is called the spectrum of the process $ X ( t) $. The spectrum of a stationary process $ X ( t) $ is concentrated on the line $ \lambda = \mu $. Under fairly general conditions, periodically correlated (or periodically non-stationary) stochastic processes $ X ( t) $( which have the property that

$$ {\mathsf E} X ( t + m T ) = {\mathsf E} X ( t ) , $$

$$ {\mathsf E} X ( t + m T ) \overline{ {X ( s + m T ) }}\; = {\mathsf E} X ( t) \overline{ {X ( s) }}\; $$

for some $ T > 0 $ and any integer $ m $) are also harmonizable. The spectra of such processes are concentrated on the set of straight lines $ \lambda = \mu + 2 \pi k / T $, $ k = 0 , \pm 1 , \pm 2 ,\dots $( see [8] or [11]).

References

[1]	K. Karhunen, "Ueber lineare Methoden in der Wahrscheinlichkeitsrechnung" Ann. Acad. Sci. Fennicae Ser. A. Math.-Phys. (I) , 37 (1947) pp. 3–79
[2]	Yu.A. Rozanov, "Spectral analysis of abstract functions" Teor. Veroyatnost. i Primenen. , 4 : 3 (1959) pp. 292–310 (In Russian)
[3]	I.M. Gel'fand, N.Ya. Vilenkin, "Generalized functions. Applications of harmonic analysis" , 4 , Acad. Press (1968) (Translated from Russian)
[4]	T. Onoyama, "Note on random distributions" Mem. Fac. Sci. Kyushu Univ. Ser. A , 13 (1959) pp. 208–213
[5a]	A.M. Yaglom, "Second-order homogeneous random fields" , Proc. 4-th Berkeley Symp. Math. Stat. Probab. , 2 , Univ. California Press (1961) pp. 593–622
[5b]	A.M. Yaglom, "Spectral representations for various classes of random functions" , Proc. 4-th All-Union Mat. Konf. 1961 , 1 , Leningrad (1963) pp. 250–273 (In Russian)
[6]	E.J. Hannan, "Group representations and applied probability" , Methuen (1965)
[7]	M.I. Yadrenko, "Spectral theory of random fields" , Optim. Software (1983) (Translated from Russian)
[8]	A.M. Yaglom, "Correlation theory of stationary and related random functions" , 1–2 , Springer (1986) (Translated from Russian)
[9]	V.S. Pugachev, "Theory of random functions and its application to control problems" , Pergamon (1965) (Translated from Russian)
[10a]	M.I. Fortus, "Method of empirical orthogonal functions and its meteorological applications" Meteorologiya i Gidrologiya , 4 (1980) pp. 113–119 (In Russian)
[10b]	H. von Stroch, G. Hannoschöck, "Statistical aspects of estimated principle components (EoFs) based on small sample sizes" Climate Appl. Meteor. , 24 (1985) pp. 716–724
[11]	S.M. Rytov, "Introduction of statistical radiophysics" , 1. Random processes , Springer (1988) (Translated from Russian)

Comments

Spectral decomposition of not necessarily stationary random functions is given in [a1].

References

[a1]	A.G. Ramm, "Random fields: estimation theory" , Longman & Wiley (1990)
[a2]	A.V. Ivanov, N.N. Leonenko, "Statistical analysis of random fields" , Kluwer (1989) (Translated from Russian)
[a3]	J.L. Doob, "Stochastic processes" , Wiley (1953)
[a4]	D.R. Cox, H.D. Miller, "The theory of stochastic processes" , Methuen (1965)
[a5]	M.S. Bartlett, "An introduction to stochastic processes" , Cambridge Univ. Press (1978)