Namespaces
Variants
Actions

Difference between revisions of "Maximum-entropy spectral estimator"

From Encyclopedia of Mathematics
Jump to: navigation, search
(Importing text file)
 
m (fix tex)
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
 +
<!--
 +
m0630901.png
 +
$#A+1 = 24 n = 0
 +
$#C+1 = 24 : ~/encyclopedia/old_files/data/M063/M.0603090 Maximum\AAhentropy spectral estimator,
 +
Automatically converted into TeX, above some diagnostics.
 +
Please remove this comment and the {{TEX|auto}} line below,
 +
if TeX found to be correct.
 +
-->
 +
 +
{{TEX|auto}}
 +
{{TEX|done}}
 +
 
''auto-regressive spectral estimator''
 
''auto-regressive spectral estimator''
  
An estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630901.png" /> for the [[Spectral density|spectral density]] <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630902.png" /> of a discrete-time stationary stochastic process such that 1) the first <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630903.png" /> values of the auto-correlations are equal to the sample auto-correlations calculated from the observational data, and 2) the [[Entropy|entropy]] of the Gaussian stochastic process with spectral density <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630904.png" /> is maximized subject to condition 1). If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630905.png" /> sample values <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630906.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630907.png" />, are known from observing a realization of a real stationary process <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630908.png" /> having spectral density <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m0630909.png" />, then the maximum-entropy spectral estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309010.png" /> is defined by the relations
+
An estimator $  f _ {q} ^ { * } ( \lambda ) $
 +
for the [[Spectral density|spectral density]] $  f ( \lambda ) $
 +
of a discrete-time stationary stochastic process such that 1) the first $  q $
 +
values of the auto-correlations are equal to the sample auto-correlations calculated from the observational data, and 2) the [[Entropy|entropy]] of the Gaussian stochastic process with spectral density $  f _ {q} ^ { * } ( \lambda ) $
 +
is maximized subject to condition 1). If $  N $
 +
sample values $  x _ {t} $,  
 +
$  t = 1 \dots N $,  
 +
are known from observing a realization of a real stationary process $  X _ {t} $
 +
having spectral density $  f ( \lambda ) $,  
 +
then the maximum-entropy spectral estimator $  f _ {q} ^ { * } ( \lambda ) $
 +
is defined by the relations
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309011.png" /></td> <td valign="top" style="width:5%;text-align:right;">(1)</td></tr></table>
+
$$ \tag{1 }
 +
\int\limits _ {- \pi } ^  \pi  \cos  k \lambda \;
 +
f _ {q} ^ { * } ( \lambda ) d \lambda  =  r _ {k}  ^ {*\ } \equiv
 +
$$
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309012.png" /></td> </tr></table>
+
$$
 +
\equiv \
 +
N  ^ {-1} \sum _ { j=1 }  ^ { N-k } x _ {j} x _ {j+k} ,\ \
 +
k = 0 \dots q ,
 +
$$
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309013.png" /></td> <td valign="top" style="width:5%;text-align:right;">(2)</td></tr></table>
+
$$ \tag{2 }
 +
\int\limits _ {- \pi } ^  \pi    \mathop{\rm log}  f _ {q} ^ { * } ( \lambda ) d \lambda  =  \max ,
 +
$$
  
where the sign <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309014.png" /> denotes  "equal by definition" . The maximum-entropy spectral estimator has the form
+
where the sign $  \equiv $
 +
denotes  "equal by definition" . The maximum-entropy spectral estimator has the form
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309015.png" /></td> <td valign="top" style="width:5%;text-align:right;">(3)</td></tr></table>
+
$$ \tag{3 }
 +
f _ {q} ^ { * } ( \lambda )  =
 +
\frac{\sigma  ^ {2} }{2 \pi | 1 + \beta _ {1}  \mathop{\rm exp} ( i \lambda ) + \dots
 +
+ \beta _ {q}  \mathop{\rm exp} ( i q \lambda ) |  ^ {2} }
 +
,
 +
$$
  
where the coefficients <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309016.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309017.png" /> are given by the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309018.png" /> equations (1) (see, e.g., [[#References|[1]]], [[#References|[9]]], [[#References|[10]]]). Formula (3) shows that the maximum-entropy spectral estimator coincides with the so-called auto-regressive spectral estimator (introduced in [[#References|[2]]], [[#References|[3]]]). The positive integer <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309019.png" /> here plays a role related to that played by the reciprocal width of a spectral window in the case of non-parametric estimation of the spectral density by periodogram smoothing (see [[Spectral window|Spectral window]]; [[Statistical problems in the theory of stochastic processes|Statistical problems in the theory of stochastic processes]]). There are several methods for estimating the optimal value of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309020.png" /> from given observations (see, for example, [[#References|[1]]], [[#References|[4]]], [[#References|[5]]], [[#References|[8]]]). The values of the coefficients <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309021.png" /> can be found using a solution of the Yule–Walker equations
+
where the coefficients $  \beta _ {1} \dots \beta _ {q} $
 +
and $  \sigma  ^ {2} $
 +
are given by the $  q + 1 $
 +
equations (1) (see, e.g., [[#References|[1]]], [[#References|[9]]], [[#References|[10]]]). Formula (3) shows that the maximum-entropy spectral estimator coincides with the so-called auto-regressive spectral estimator (introduced in [[#References|[2]]], [[#References|[3]]]). The positive integer $  q $
 +
here plays a role related to that played by the reciprocal width of a spectral window in the case of non-parametric estimation of the spectral density by periodogram smoothing (see [[Spectral window|Spectral window]]; [[Statistical problems in the theory of stochastic processes|Statistical problems in the theory of stochastic processes]]). There are several methods for estimating the optimal value of $  q $
 +
from given observations (see, for example, [[#References|[1]]], [[#References|[4]]], [[#References|[5]]], [[#References|[8]]]). The values of the coefficients $  \beta _ {1} \dots \beta _ {q} , \sigma  ^ {2} $
 +
can be found using a solution of the Yule–Walker equations
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309022.png" /></td> <td valign="top" style="width:5%;text-align:right;">(4)</td></tr></table>
+
$$ \tag{4 }
 +
r _ {k}  ^ {*} + \sum _ { j=1 } ^ { q }  \beta _ {j} r _ {| k - j | }  ^ {*}  = 0 ,\  k = 1 \dots q ,
 +
$$
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309023.png" /></td> <td valign="top" style="width:5%;text-align:right;">(5)</td></tr></table>
+
$$ \tag{5 }
 +
r _ {0}  ^ {*} + \sum _ { j=1 } ^ { q }  \beta _ {j} r _ {j}  ^ {*}  = \sigma  ^ {2} ;
 +
$$
  
 
there are also other, numerically more convenient, methods for calculating these coefficients (see, e.g., [[#References|[1]]], [[#References|[4]]]–[[#References|[6]]], [[#References|[10]]]).
 
there are also other, numerically more convenient, methods for calculating these coefficients (see, e.g., [[#References|[1]]], [[#References|[4]]]–[[#References|[6]]], [[#References|[10]]]).
  
In the case of small sample size or spectral densities of complex form, maximum-entropy spectral estimators and parametric spectral estimators (cf. [[Spectral estimator, parametric|Spectral estimator, parametric]]), which generalize them, possess definite advantages over non-parametric estimators of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m063/m063090/m06309024.png" />: they usually have a more regular form and possess better resolving power, that is, they permit one to better distinguish close peaks of the graph of the spectral density (see [[#References|[1]]], [[#References|[4]]]–[[#References|[7]]]). Therefore maximum-entropy spectral estimators are widely used in the applied [[Spectral analysis of a stationary stochastic process|spectral analysis of a stationary stochastic process]].
+
In the case of small sample size or spectral densities of complex form, maximum-entropy spectral estimators and parametric spectral estimators (cf. [[Spectral estimator, parametric|Spectral estimator, parametric]]), which generalize them, possess definite advantages over non-parametric estimators of $  f ( \lambda ) $:  
 +
they usually have a more regular form and possess better resolving power, that is, they permit one to better distinguish close peaks of the graph of the spectral density (see [[#References|[1]]], [[#References|[4]]]–[[#References|[7]]]). Therefore maximum-entropy spectral estimators are widely used in the applied [[Spectral analysis of a stationary stochastic process|spectral analysis of a stationary stochastic process]].
  
 
====References====
 
====References====
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  D.G. Childers (ed.) , ''Modern spectrum analysis'' , IEEE  (1978)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  E. Parzen,  "An approach to empirical time series analysis"  ''Radio Sci.'' , '''68'''  (1964)  pp. 937–951</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  H. Akaike,  "Power spectrum estimation through autoregressive model fitting"  ''Ann. Inst. Stat. Math.'' , '''21''' :  3  (1969)  pp. 407–419</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  S.S. Haykin (ed.) , ''Nonlinear methods of spectral analysis'' , Springer  (1979)</TD></TR><TR><TD valign="top">[5]</TD> <TD valign="top">  S.M. Kay,  S.L. Marpl,  "Spectrum analysis—a modern perspective"  ''Proc. IEEE'' , '''69''' :  11  (1981)  pp. 1380–1419</TD></TR><TR><TD valign="top">[6]</TD> <TD valign="top">  "Spectral estimation"  ''Proc. IEEE'' , '''70''' :  9  (1982)  ((Special Issue))</TD></TR><TR><TD valign="top">[7]</TD> <TD valign="top">  V.F. Pisarenko,  "Sampling properties of maximum entropy spectral estimation" , ''Numerical Seismology'' , Moscow  (1977)  pp. 118–149  (In Russian)</TD></TR><TR><TD valign="top">[8]</TD> <TD valign="top">  J.G. de Gooyer,  B. Abraham,  A. Gould,  L. Robinson,  "Methods for determining the order of an autoregressive-moving average process: A survey"  ''Internat. Stat. Rev.'' , '''55'''  (1985)  pp. 301–329</TD></TR><TR><TD valign="top">[9]</TD> <TD valign="top">  M.B. Priestley,  "Spectral analysis and time series" , '''1–2''' , Acad. Press  (1981)</TD></TR><TR><TD valign="top">[10]</TD> <TD valign="top">  A. Papoulis,  "Probability, random variables and stochastic processes" , McGraw-Hill  (1984)</TD></TR></table>
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  D.G. Childers (ed.) , ''Modern spectrum analysis'' , IEEE  (1978)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  E. Parzen,  "An approach to empirical time series analysis"  ''Radio Sci.'' , '''68'''  (1964)  pp. 937–951</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  H. Akaike,  "Power spectrum estimation through autoregressive model fitting"  ''Ann. Inst. Stat. Math.'' , '''21''' :  3  (1969)  pp. 407–419</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  S.S. Haykin (ed.) , ''Nonlinear methods of spectral analysis'' , Springer  (1979)</TD></TR><TR><TD valign="top">[5]</TD> <TD valign="top">  S.M. Kay,  S.L. Marpl,  "Spectrum analysis—a modern perspective"  ''Proc. IEEE'' , '''69''' :  11  (1981)  pp. 1380–1419</TD></TR><TR><TD valign="top">[6]</TD> <TD valign="top">  "Spectral estimation"  ''Proc. IEEE'' , '''70''' :  9  (1982)  ((Special Issue))</TD></TR><TR><TD valign="top">[7]</TD> <TD valign="top">  V.F. Pisarenko,  "Sampling properties of maximum entropy spectral estimation" , ''Numerical Seismology'' , Moscow  (1977)  pp. 118–149  (In Russian)</TD></TR><TR><TD valign="top">[8]</TD> <TD valign="top">  J.G. de Gooyer,  B. Abraham,  A. Gould,  L. Robinson,  "Methods for determining the order of an autoregressive-moving average process: A survey"  ''Internat. Stat. Rev.'' , '''55'''  (1985)  pp. 301–329</TD></TR><TR><TD valign="top">[9]</TD> <TD valign="top">  M.B. Priestley,  "Spectral analysis and time series" , '''1–2''' , Acad. Press  (1981)</TD></TR><TR><TD valign="top">[10]</TD> <TD valign="top">  A. Papoulis,  "Probability, random variables and stochastic processes" , McGraw-Hill  (1984)</TD></TR></table>

Latest revision as of 18:22, 14 January 2021


auto-regressive spectral estimator

An estimator $ f _ {q} ^ { * } ( \lambda ) $ for the spectral density $ f ( \lambda ) $ of a discrete-time stationary stochastic process such that 1) the first $ q $ values of the auto-correlations are equal to the sample auto-correlations calculated from the observational data, and 2) the entropy of the Gaussian stochastic process with spectral density $ f _ {q} ^ { * } ( \lambda ) $ is maximized subject to condition 1). If $ N $ sample values $ x _ {t} $, $ t = 1 \dots N $, are known from observing a realization of a real stationary process $ X _ {t} $ having spectral density $ f ( \lambda ) $, then the maximum-entropy spectral estimator $ f _ {q} ^ { * } ( \lambda ) $ is defined by the relations

$$ \tag{1 } \int\limits _ {- \pi } ^ \pi \cos k \lambda \; f _ {q} ^ { * } ( \lambda ) d \lambda = r _ {k} ^ {*\ } \equiv $$

$$ \equiv \ N ^ {-1} \sum _ { j=1 } ^ { N-k } x _ {j} x _ {j+k} ,\ \ k = 0 \dots q , $$

$$ \tag{2 } \int\limits _ {- \pi } ^ \pi \mathop{\rm log} f _ {q} ^ { * } ( \lambda ) d \lambda = \max , $$

where the sign $ \equiv $ denotes "equal by definition" . The maximum-entropy spectral estimator has the form

$$ \tag{3 } f _ {q} ^ { * } ( \lambda ) = \frac{\sigma ^ {2} }{2 \pi | 1 + \beta _ {1} \mathop{\rm exp} ( i \lambda ) + \dots + \beta _ {q} \mathop{\rm exp} ( i q \lambda ) | ^ {2} } , $$

where the coefficients $ \beta _ {1} \dots \beta _ {q} $ and $ \sigma ^ {2} $ are given by the $ q + 1 $ equations (1) (see, e.g., [1], [9], [10]). Formula (3) shows that the maximum-entropy spectral estimator coincides with the so-called auto-regressive spectral estimator (introduced in [2], [3]). The positive integer $ q $ here plays a role related to that played by the reciprocal width of a spectral window in the case of non-parametric estimation of the spectral density by periodogram smoothing (see Spectral window; Statistical problems in the theory of stochastic processes). There are several methods for estimating the optimal value of $ q $ from given observations (see, for example, [1], [4], [5], [8]). The values of the coefficients $ \beta _ {1} \dots \beta _ {q} , \sigma ^ {2} $ can be found using a solution of the Yule–Walker equations

$$ \tag{4 } r _ {k} ^ {*} + \sum _ { j=1 } ^ { q } \beta _ {j} r _ {| k - j | } ^ {*} = 0 ,\ k = 1 \dots q , $$

$$ \tag{5 } r _ {0} ^ {*} + \sum _ { j=1 } ^ { q } \beta _ {j} r _ {j} ^ {*} = \sigma ^ {2} ; $$

there are also other, numerically more convenient, methods for calculating these coefficients (see, e.g., [1], [4][6], [10]).

In the case of small sample size or spectral densities of complex form, maximum-entropy spectral estimators and parametric spectral estimators (cf. Spectral estimator, parametric), which generalize them, possess definite advantages over non-parametric estimators of $ f ( \lambda ) $: they usually have a more regular form and possess better resolving power, that is, they permit one to better distinguish close peaks of the graph of the spectral density (see [1], [4][7]). Therefore maximum-entropy spectral estimators are widely used in the applied spectral analysis of a stationary stochastic process.

References

[1] D.G. Childers (ed.) , Modern spectrum analysis , IEEE (1978)
[2] E. Parzen, "An approach to empirical time series analysis" Radio Sci. , 68 (1964) pp. 937–951
[3] H. Akaike, "Power spectrum estimation through autoregressive model fitting" Ann. Inst. Stat. Math. , 21 : 3 (1969) pp. 407–419
[4] S.S. Haykin (ed.) , Nonlinear methods of spectral analysis , Springer (1979)
[5] S.M. Kay, S.L. Marpl, "Spectrum analysis—a modern perspective" Proc. IEEE , 69 : 11 (1981) pp. 1380–1419
[6] "Spectral estimation" Proc. IEEE , 70 : 9 (1982) ((Special Issue))
[7] V.F. Pisarenko, "Sampling properties of maximum entropy spectral estimation" , Numerical Seismology , Moscow (1977) pp. 118–149 (In Russian)
[8] J.G. de Gooyer, B. Abraham, A. Gould, L. Robinson, "Methods for determining the order of an autoregressive-moving average process: A survey" Internat. Stat. Rev. , 55 (1985) pp. 301–329
[9] M.B. Priestley, "Spectral analysis and time series" , 1–2 , Acad. Press (1981)
[10] A. Papoulis, "Probability, random variables and stochastic processes" , McGraw-Hill (1984)
How to Cite This Entry:
Maximum-entropy spectral estimator. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Maximum-entropy_spectral_estimator&oldid=17257
This article was adapted from an original article by A.M. Yaglom (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article