Spectral decomposition of a random function

spectral representation of a random function

A representation of a random function (in particular, of a stochastic process) by a series or integral with respect to some special system of functions, such that the coefficients in this expansion are pairwise uncorrelated random variables. A wide class of spectral representations of complex-valued random functions , , with zero mean value (that is, such that ) can be written in the form

(1)

where is some set with a given system of "measurable subsets" (that is, a measurable space); , , , is a system of complex-valued functions of , depending on a parameter ; is an orthogonal random measure on (with uncorrelated values, so that for any two disjoint measurable subsets and ); and the integral on the right-hand side can be defined either as the mean-square limit of the corresponding sequence of integral Cauchy sums (), or, more generally, as a "Lebesgue integral with respect to the measure Zdl" (see , for example). According to the general Karhunen spectral representation theorem, a spectral representation (1) exists for a random function if and only if the corresponding correlation function can be written in the form

where is a non-negative measure on .

The best known class of spectral representations of random functions are the representations of stationary stochastic processes as Fourier–Stieltjes integrals,

(2)

where is a random function of with uncorrelated increments and is either the real line , when the time is continuous, or the interval , when is discrete (and takes integral values). The existence of such a spectral decomposition follows from the general theorem of Khinchin (or Wiener–Khinchin) on the integral representation of the correlation function (see Stationary stochastic process). This shows that any stationary stochastic process can be regarded as the superposition of mutually uncorrelated harmonic oscillations of various frequencies and with random phases and amplitudes. A spectral decomposition of similar form, but with -dimensional planar waves in place of harmonic oscillations, also exists for homogeneous random fields defined on a Euclidean -dimensional space , or on the lattice of integer points in . In the case of a generalized stationary stochastic process, consider a linear functional on the space of infinitely-differentiable functions of compact support satisfying the conditions

for all real , where . The functional can be written in the form

(3)

where

is the Fourier transform of . Formula (3) follows from the fact that

can be written in the form

where the function is a monotone non-decreasing spectral distribution function such that

for some non-negative integer (see ). If one takes the space of functions to be some special space of entire analytic functions, then one arrives at a generalized stationary stochastic process with exponentially increasing spectral function (see ).

Spectral decompositions of special form also occur for homogeneous random fields on groups and on homogeneous spaces . This is a consequence of Karhunen's spectral decomposition theorem together with certain well-known results on the general form of positive-definite functions (or kernels, which are functions in two variables) on the sets and . In particular, for a homogeneous field on an arbitrary locally compact Abelian group , the spectral representation of has the form (1), where the role of the functions is played by the characters of , and the domain of integration is the corresponding character group (see, , , for example). Spectral representations of more complicated form also occur under fairly general conditions for homogeneous fields on non-commutative topological groups (see ). Finally, in the case of homogeneous fields on homogeneous spaces , the spectral decomposition of a field involves the spherical functions (spherical harmonics) in the space , and the corresponding spectral representation of the correlation function involves the zonal spherical functions (see , ). In particular, a general homogeneous field on the sphere in the three-dimensional space admits a spectral representation of the form

(4)

where

are ordinary spherical harmonics and the random variables are such that , where is the Kronecker delta. Corresponding to formula (4) there is an expression for the correlation function of the form

where is the angular distance between the points and , and

where the are the Legendre polynomials. Similarly, if (where are polar coordinates) is a homogeneous and isotropic field in the plane (so that , where is the Euclidean distance between the points and ), then the spectral representation of can be written in the form

(5)

where is the Bessel function of order (cf Bessel functions). Here the are random functions with uncorrelated increments such that

where

and is a non-negative measure on the semi-axis . Corresponding to the spectral representation (5), there is the following expression for the correlation function :

For further examples of spectral representations of homogeneous fields see –.

Spectral representations of random functions exist not only for stationary stochastic processes and homogeneous fields. For example, if is an arbitrary stochastic process on the interval with correlation function

which is continuous in both arguments, then by virtue of Mercer's theorem in the theory of integral equations (cf. Mercer theorem) and Karhunen's spectral representation theorem, admits a spectral representation of the form

(6)

where the and the , are the eigenfunctions and eigenvalues of the integral operator on the function space with kernel and . The spectral representation (6) of a stochastic process , defined on a finite interval, is the continuous analogue of the decomposition of a random vector into its principal components, which is often used in multivariate statistical analysis. It was obtained independently by a number of scientists (see , , for example) and is most frequently called the Karhunen–Loève expansion. Spectral representations of the form (6) are widely used in many applications, in particular in the theory of automatic control, where (6) and some related representations are often called canonical representations of stochastic processes (see ), and in meteorology and geophysics, where the term "method of empirical orthogonal functionsmethod of empirical orthogonal functions" is usually used, since the eigenfunctions in practice must be approximately determined by empirical data (see , ).

A spectral representation of a random function , , can sometimes also mean a general representation of the form

(without the requirement that is a random measure with uncorrelated values) into a certain standard (sufficiently simple) complete system of functions . This is most common in the case of decompositions of stochastic processes with continuous time into functions , so that

reduces to . It follows from

that, in general, can be written in the form

(7)

where is the complex-valued measure on the -plane defined by the relation

Conversely, it can be easily shown that the fact that can be written in the form (7) implies that there is a spectral representation

(see, for example, [2]). Stochastic processes admitting a spectral representation , where does not necessarily have uncorrelated increments, are called harmonizable stochastic processes. In this case, the complex measure is called the spectral measure of , and the set of points of the -plane not having a neighbourhood of spectral measure zero is called the spectrum of the process . The spectrum of a stationary process is concentrated on the line . Under fairly general conditions, periodically correlated (or periodically non-stationary) stochastic processes (which have the property that

for some and any integer ) are also harmonizable. The spectra of such processes are concentrated on the set of straight lines , (see [8] or [11]).

References

[1]	K. Karhunen, "Ueber lineare Methoden in der Wahrscheinlichkeitsrechnung" Ann. Acad. Sci. Fennicae Ser. A. Math.-Phys. (I) , 37 (1947) pp. 3–79
[2]	Yu.A. Rozanov, "Spectral analysis of abstract functions" Teor. Veroyatnost. i Primenen. , 4 : 3 (1959) pp. 292–310 (In Russian)
[3]	I.M. Gel'fand, N.Ya. Vilenkin, "Generalized functions. Applications of harmonic analysis" , 4 , Acad. Press (1968) (Translated from Russian)
[4]	T. Onoyama, "Note on random distributions" Mem. Fac. Sci. Kyushu Univ. Ser. A , 13 (1959) pp. 208–213
[5a]	A.M. Yaglom, "Second-order homogeneous random fields" , Proc. 4-th Berkeley Symp. Math. Stat. Probab. , 2 , Univ. California Press (1961) pp. 593–622
[5b]	A.M. Yaglom, "Spectral representations for various classes of random functions" , Proc. 4-th All-Union Mat. Konf. 1961 , 1 , Leningrad (1963) pp. 250–273 (In Russian)
[6]	E.J. Hannan, "Group representations and applied probability" , Methuen (1965)
[7]	M.I. Yadrenko, "Spectral theory of random fields" , Optim. Software (1983) (Translated from Russian)
[8]	A.M. Yaglom, "Correlation theory of stationary and related random functions" , 1–2 , Springer (1986) (Translated from Russian)
[9]	V.S. Pugachev, "Theory of random functions and its application to control problems" , Pergamon (1965) (Translated from Russian)
[10a]	M.I. Fortus, "Method of empirical orthogonal functions and its meteorological applications" Meteorologiya i Gidrologiya , 4 (1980) pp. 113–119 (In Russian)
[10b]	H. von Stroch, G. Hannoschöck, "Statistical aspects of estimated principle components (EoFs) based on small sample sizes" Climate Appl. Meteor. , 24 (1985) pp. 716–724
[11]	S.M. Rytov, "Introduction of statistical radiophysics" , 1. Random processes , Springer (1988) (Translated from Russian)

Comments

Spectral decomposition of not necessarily stationary random functions is given in [a1].

References

[a1]	A.G. Ramm, "Random fields: estimation theory" , Longman & Wiley (1990)
[a2]	A.V. Ivanov, N.N. Leonenko, "Statistical analysis of random fields" , Kluwer (1989) (Translated from Russian)
[a3]	J.L. Doob, "Stochastic processes" , Wiley (1953)
[a4]	D.R. Cox, H.D. Miller, "The theory of stochastic processes" , Methuen (1965)
[a5]	M.S. Bartlett, "An introduction to stochastic processes" , Cambridge Univ. Press (1978)