View source for Maximum-entropy spectral estimator

<!--
m0630901.png
$#A+1 = 24 n = 0
$#C+1 = 24 : ~/encyclopedia/old_files/data/M063/M.0603090 Maximum\AAhentropy spectral estimator,
Automatically converted into TeX, above some diagnostics.
Please remove this comment and the {{TEX|auto}} line below,
if TeX found to be correct.
-->

{{TEX|auto}}
{{TEX|done}}

''auto-regressive spectral estimator''

An estimator  $  f _ {q} ^ { * } ( \lambda ) $
for the [[Spectral density|spectral density]]  $  f ( \lambda ) $
of a discrete-time stationary stochastic process such that 1) the first  $  q $
values of the auto-correlations are equal to the sample auto-correlations calculated from the observational data, and 2) the [[Entropy|entropy]] of the Gaussian stochastic process with spectral density  $  f _ {q} ^ { * } ( \lambda ) $
is maximized subject to condition 1). If  $  N $
sample values  $  x _ {t} $, 
$  t = 1 \dots N $, 
are known from observing a realization of a real stationary process  $  X _ {t} $
having spectral density  $  f ( \lambda ) $, 
then the maximum-entropy spectral estimator  $  f _ {q} ^ { * } ( \lambda ) $
is defined by the relations

$$ \tag{1 }
\int\limits _ {- \pi } ^  \pi   \cos  k \lambda
f _ {q} ^ { * } ( \lambda )  d \lambda  =  r _ {k}  ^ {*\ } \equiv
$$

$$ 
\equiv \ 
N  ^ {-} 1 \sum _ { j= } 1 ^ { N- }  k x _ {j} x _ {j+} k ,\ \ 
k = 0 \dots q ,
$$

$$ \tag{2 }
\int\limits _ {- \pi } ^  \pi    \mathop{\rm log}  f _ {q} ^ { * } ( \lambda )  d \lambda  =  \max ,
$$

where the sign  $  \equiv $
denotes  "equal by definition" . The maximum-entropy spectral estimator has the form

$$ \tag{3 }
f _ {q} ^ { * } ( \lambda )  =  
\frac{\sigma  ^ {2} }{2 \pi | 1 + \beta _ {1}  \mathop{\rm exp} ( i \lambda ) + \dots
+ \beta _ {q}  \mathop{\rm exp} ( i q \lambda ) |  ^ {2} }
 ,
$$

where the coefficients  $  \beta _ {1} \dots \beta _ {q} $
and  $  \sigma  ^ {2} $
are given by the  $  q + 1 $
equations (1) (see, e.g., [[#References|[1]]], [[#References|[9]]], [[#References|[10]]]). Formula (3) shows that the maximum-entropy spectral estimator coincides with the so-called auto-regressive spectral estimator (introduced in [[#References|[2]]], [[#References|[3]]]). The positive integer  $  q $
here plays a role related to that played by the reciprocal width of a spectral window in the case of non-parametric estimation of the spectral density by periodogram smoothing (see [[Spectral window|Spectral window]]; [[Statistical problems in the theory of stochastic processes|Statistical problems in the theory of stochastic processes]]). There are several methods for estimating the optimal value of  $  q $
from given observations (see, for example, [[#References|[1]]], [[#References|[4]]], [[#References|[5]]], [[#References|[8]]]). The values of the coefficients  $  \beta _ {1} \dots \beta _ {q} , \sigma  ^ {2} $
can be found using a solution of the Yule–Walker equations

$$ \tag{4 }
r _ {k}  ^ {*} + \sum _ { j= } 1 ^ { q }  \beta _ {j} r _ {| k - j | }   ^ {*}  =  0 ,\  k = 1 \dots q ,
$$

$$ \tag{5 }
r _ {0}  ^ {*} + \sum _ { j= } 1 ^ { q }  \beta _ {j} r _ {j}  ^ {*}  =  \sigma  ^ {2} ;
$$

there are also other, numerically more convenient, methods for calculating these coefficients (see, e.g., [[#References|[1]]], [[#References|[4]]]–[[#References|[6]]], [[#References|[10]]]).

In the case of small sample size or spectral densities of complex form, maximum-entropy spectral estimators and parametric spectral estimators (cf. [[Spectral estimator, parametric|Spectral estimator, parametric]]), which generalize them, possess definite advantages over non-parametric estimators of  $  f ( \lambda ) $: 
they usually have a more regular form and possess better resolving power, that is, they permit one to better distinguish close peaks of the graph of the spectral density (see [[#References|[1]]], [[#References|[4]]]–[[#References|[7]]]). Therefore maximum-entropy spectral estimators are widely used in the applied [[Spectral analysis of a stationary stochastic process|spectral analysis of a stationary stochastic process]].

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  D.G. Childers (ed.) , ''Modern spectrum analysis'' , IEEE  (1978)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  E. Parzen,   "An approach to empirical time series analysis"  ''Radio Sci.'' , '''68'''  (1964)  pp. 937–951</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  H. Akaike,   "Power spectrum estimation through autoregressive model fitting"  ''Ann. Inst. Stat. Math.'' , '''21''' :  3  (1969)  pp. 407–419</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  S.S. Haykin (ed.) , ''Nonlinear methods of spectral analysis'' , Springer  (1979)</TD></TR><TR><TD valign="top">[5]</TD> <TD valign="top">  S.M. Kay,   S.L. Marpl,   "Spectrum analysis—a modern perspective"  ''Proc. IEEE'' , '''69''' :  11  (1981)  pp. 1380–1419</TD></TR><TR><TD valign="top">[6]</TD> <TD valign="top">  "Spectral estimation"  ''Proc. IEEE'' , '''70''' :  9  (1982)  ((Special Issue))</TD></TR><TR><TD valign="top">[7]</TD> <TD valign="top">  V.F. Pisarenko,   "Sampling properties of maximum entropy spectral estimation" , ''Numerical Seismology'' , Moscow  (1977)  pp. 118–149  (In Russian)</TD></TR><TR><TD valign="top">[8]</TD> <TD valign="top">  J.G. de Gooyer,   B. Abraham,   A. Gould,   L. Robinson,   "Methods for determining the order of an autoregressive-moving average process: A survey"  ''Internat. Stat. Rev.'' , '''55'''  (1985)  pp. 301–329</TD></TR><TR><TD valign="top">[9]</TD> <TD valign="top">  M.B. Priestley,   "Spectral analysis and time series" , '''1–2''' , Acad. Press  (1981)</TD></TR><TR><TD valign="top">[10]</TD> <TD valign="top">  A. Papoulis,   "Probability, random variables and stochastic processes" , McGraw-Hill  (1984)</TD></TR></table>