Maximum-entropy spectral estimator

auto-regressive spectral estimator

An estimator $ f _ {q} ^ { * } ( \lambda ) $ for the spectral density $ f ( \lambda ) $ of a discrete-time stationary stochastic process such that 1) the first $ q $ values of the auto-correlations are equal to the sample auto-correlations calculated from the observational data, and 2) the entropy of the Gaussian stochastic process with spectral density $ f _ {q} ^ { * } ( \lambda ) $ is maximized subject to condition 1). If $ N $ sample values $ x _ {t} $, $ t = 1 \dots N $, are known from observing a realization of a real stationary process $ X _ {t} $ having spectral density $ f ( \lambda ) $, then the maximum-entropy spectral estimator $ f _ {q} ^ { * } ( \lambda ) $ is defined by the relations

$$ \tag{1 } \int\limits _ {- \pi } ^ \pi \cos k \lambda \; f _ {q} ^ { * } ( \lambda ) d \lambda = r _ {k} ^ {*\ } \equiv $$

$$ \equiv \ N ^ {-1} \sum _ { j=1 } ^ { N-k } x _ {j} x _ {j+k} ,\ \ k = 0 \dots q , $$

$$ \tag{2 } \int\limits _ {- \pi } ^ \pi \mathop{\rm log} f _ {q} ^ { * } ( \lambda ) d \lambda = \max , $$

where the sign $ \equiv $ denotes "equal by definition" . The maximum-entropy spectral estimator has the form

$$ \tag{3 } f _ {q} ^ { * } ( \lambda ) = \frac{\sigma ^ {2} }{2 \pi | 1 + \beta _ {1} \mathop{\rm exp} ( i \lambda ) + \dots + \beta _ {q} \mathop{\rm exp} ( i q \lambda ) | ^ {2} } , $$

where the coefficients $ \beta _ {1} \dots \beta _ {q} $ and $ \sigma ^ {2} $ are given by the $ q + 1 $ equations (1) (see, e.g., [1], [9], [10]). Formula (3) shows that the maximum-entropy spectral estimator coincides with the so-called auto-regressive spectral estimator (introduced in [2], [3]). The positive integer $ q $ here plays a role related to that played by the reciprocal width of a spectral window in the case of non-parametric estimation of the spectral density by periodogram smoothing (see Spectral window; Statistical problems in the theory of stochastic processes). There are several methods for estimating the optimal value of $ q $ from given observations (see, for example, [1], [4], [5], [8]). The values of the coefficients $ \beta _ {1} \dots \beta _ {q} , \sigma ^ {2} $ can be found using a solution of the Yule–Walker equations

$$ \tag{4 } r _ {k} ^ {*} + \sum _ { j=1 } ^ { q } \beta _ {j} r _ {| k - j | } ^ {*} = 0 ,\ k = 1 \dots q , $$

$$ \tag{5 } r _ {0} ^ {*} + \sum _ { j=1 } ^ { q } \beta _ {j} r _ {j} ^ {*} = \sigma ^ {2} ; $$

there are also other, numerically more convenient, methods for calculating these coefficients (see, e.g., [1], [4]–[6], [10]).

In the case of small sample size or spectral densities of complex form, maximum-entropy spectral estimators and parametric spectral estimators (cf. Spectral estimator, parametric), which generalize them, possess definite advantages over non-parametric estimators of $ f ( \lambda ) $: they usually have a more regular form and possess better resolving power, that is, they permit one to better distinguish close peaks of the graph of the spectral density (see [1], [4]–[7]). Therefore maximum-entropy spectral estimators are widely used in the applied spectral analysis of a stationary stochastic process.

References

[1]	D.G. Childers (ed.) , Modern spectrum analysis , IEEE (1978)
[2]	E. Parzen, "An approach to empirical time series analysis" Radio Sci. , 68 (1964) pp. 937–951
[3]	H. Akaike, "Power spectrum estimation through autoregressive model fitting" Ann. Inst. Stat. Math. , 21 : 3 (1969) pp. 407–419
[4]	S.S. Haykin (ed.) , Nonlinear methods of spectral analysis , Springer (1979)
[5]	S.M. Kay, S.L. Marpl, "Spectrum analysis—a modern perspective" Proc. IEEE , 69 : 11 (1981) pp. 1380–1419
[6]	"Spectral estimation" Proc. IEEE , 70 : 9 (1982) ((Special Issue))
[7]	V.F. Pisarenko, "Sampling properties of maximum entropy spectral estimation" , Numerical Seismology , Moscow (1977) pp. 118–149 (In Russian)
[8]	J.G. de Gooyer, B. Abraham, A. Gould, L. Robinson, "Methods for determining the order of an autoregressive-moving average process: A survey" Internat. Stat. Rev. , 55 (1985) pp. 301–329
[9]	M.B. Priestley, "Spectral analysis and time series" , 1–2 , Acad. Press (1981)
[10]	A. Papoulis, "Probability, random variables and stochastic processes" , McGraw-Hill (1984)