Namespaces
Variants
Actions

Difference between revisions of "Information, transmission rate of"

From Encyclopedia of Mathematics
Jump to: navigation, search
(Importing text file)
 
m (tex encoded by computer)
 
Line 1: Line 1:
 +
<!--
 +
i0511601.png
 +
$#A+1 = 17 n = 0
 +
$#C+1 = 17 : ~/encyclopedia/old_files/data/I051/I.0501160 Information, transmission rate of
 +
Automatically converted into TeX, above some diagnostics.
 +
Please remove this comment and the {{TEX|auto}} line below,
 +
if TeX found to be correct.
 +
-->
 +
 +
{{TEX|auto}}
 +
{{TEX|done}}
 +
 
A quantity characterizing the amount of information that is contained in the output signal of a [[Communication channel|communication channel]] relative to the input signal, calculated in a unit of time (cf. [[Information, amount of|Information, amount of]]). If
 
A quantity characterizing the amount of information that is contained in the output signal of a [[Communication channel|communication channel]] relative to the input signal, calculated in a unit of time (cf. [[Information, amount of|Information, amount of]]). If
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511601.png" /></td> </tr></table>
+
$$
 +
\eta  = \{ {\eta ( t) } : {- \infty < t < \infty } \}
 +
,\ \
 +
\widetilde \eta    = \{ {\widetilde \eta  ( t) } : {- \infty < t < \infty } \}
 +
$$
  
 
are stochastic processes in discrete or continuous time, being the input and output signals of a communication channel, then the quantity
 
are stochastic processes in discrete or continuous time, being the input and output signals of a communication channel, then the quantity
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511602.png" /></td> <td valign="top" style="width:5%;text-align:right;">(*)</td></tr></table>
+
$$ \tag{* }
 +
= \lim\limits _ {T - t \rightarrow \infty } \
 +
 
 +
\frac{1}{T-}
 +
1
 +
I ( \eta _ {t}  ^ {T} , \widetilde \eta  {} _ {t}  ^ {T} ) ,
 +
$$
 +
 
 +
is the transmission rate of information (if the limit exists). Here  $  I ( \cdot , \cdot ) $
 +
is the amount of information,  $  \eta _ {t}  ^ {T} = \{ {\eta ( s) } : {t < s \leq  T } \} $
 +
is the segment  $  [ t , T ] $
 +
of  $  \eta $
 +
and  $  \widetilde \eta  {} _ {t}  ^ {T} $
 +
is analogously defined. The existence of the limit in (*) has been proved for the large class of channels in which the signals  $  \eta $
 +
and  $  \widetilde \eta  $
 +
are stationary and stationarily-related stochastic processes. An explicit computation of the transmission rate of information is possible, in particular, for a [[Memoryless channel|memoryless channel]] and a [[Gaussian channel|Gaussian channel]]. E.g., for a Gaussian channel, whose signals  $  \eta $
 +
and  $  \widetilde \eta  $
 +
are Gaussian stationary processes forming a joint Gaussian stationary pair of processes, the transmission rate of information is given by
 +
 
 +
$$
 +
= -
 +
\frac{1}{2}
 +
\int\limits _ {- \infty } ^  \infty 
 +
\mathop{\rm log} \
 +
\left (
 +
1 -
  
is the transmission rate of information (if the limit exists). Here <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511603.png" /> is the amount of information, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511604.png" /> is the segment <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511605.png" /> of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511606.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511607.png" /> is analogously defined. The existence of the limit in (*) has been proved for the large class of channels in which the signals <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511608.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i0511609.png" /> are stationary and stationarily-related stochastic processes. An explicit computation of the transmission rate of information is possible, in particular, for a [[Memoryless channel|memoryless channel]] and a [[Gaussian channel|Gaussian channel]]. E.g., for a Gaussian channel, whose signals <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116010.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116011.png" /> are Gaussian stationary processes forming a joint Gaussian stationary pair of processes, the transmission rate of information is given by
+
\frac{| f _ {\eta \widetilde \eta  }  ( \lambda ) |  ^ {2} }{f _ {\eta \eta }  ( \lambda ) f _ {\widetilde \eta  \widetilde \eta  }  ( \lambda ) }
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116012.png" /></td> </tr></table>
+
\right ) \
 +
d \lambda ,
 +
$$
  
where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116013.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116014.png" /> are the spectral densities of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116015.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116016.png" />, respectively, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/i/i051/i051160/i05116017.png" /> is their joint [[Spectral density|spectral density]].
+
where $  f _ {\eta \eta }  ( \lambda ) $
 +
and $  f _ {\widetilde \eta  \widetilde \eta  }  ( \lambda ) $
 +
are the spectral densities of $  \eta $
 +
and $  \widetilde \eta  $,  
 +
respectively, and $  f _ {\eta \widetilde \eta  }  ( \lambda ) $
 +
is their joint [[Spectral density|spectral density]].
  
 
====References====
 
====References====
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  R. Gallagher,  "Information theory and reliable communication" , Wiley  (1968)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  M.S. Pinsker,  "Information and informational stability of random variables and processes" , Holden-Day  (1964)  (Translated from Russian)</TD></TR></table>
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  R. Gallagher,  "Information theory and reliable communication" , Wiley  (1968)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  M.S. Pinsker,  "Information and informational stability of random variables and processes" , Holden-Day  (1964)  (Translated from Russian)</TD></TR></table>

Latest revision as of 22:12, 5 June 2020


A quantity characterizing the amount of information that is contained in the output signal of a communication channel relative to the input signal, calculated in a unit of time (cf. Information, amount of). If

$$ \eta = \{ {\eta ( t) } : {- \infty < t < \infty } \} ,\ \ \widetilde \eta = \{ {\widetilde \eta ( t) } : {- \infty < t < \infty } \} $$

are stochastic processes in discrete or continuous time, being the input and output signals of a communication channel, then the quantity

$$ \tag{* } R = \lim\limits _ {T - t \rightarrow \infty } \ \frac{1}{T-} 1 I ( \eta _ {t} ^ {T} , \widetilde \eta {} _ {t} ^ {T} ) , $$

is the transmission rate of information (if the limit exists). Here $ I ( \cdot , \cdot ) $ is the amount of information, $ \eta _ {t} ^ {T} = \{ {\eta ( s) } : {t < s \leq T } \} $ is the segment $ [ t , T ] $ of $ \eta $ and $ \widetilde \eta {} _ {t} ^ {T} $ is analogously defined. The existence of the limit in (*) has been proved for the large class of channels in which the signals $ \eta $ and $ \widetilde \eta $ are stationary and stationarily-related stochastic processes. An explicit computation of the transmission rate of information is possible, in particular, for a memoryless channel and a Gaussian channel. E.g., for a Gaussian channel, whose signals $ \eta $ and $ \widetilde \eta $ are Gaussian stationary processes forming a joint Gaussian stationary pair of processes, the transmission rate of information is given by

$$ R = - \frac{1}{2} \int\limits _ {- \infty } ^ \infty \mathop{\rm log} \ \left ( 1 - \frac{| f _ {\eta \widetilde \eta } ( \lambda ) | ^ {2} }{f _ {\eta \eta } ( \lambda ) f _ {\widetilde \eta \widetilde \eta } ( \lambda ) } \right ) \ d \lambda , $$

where $ f _ {\eta \eta } ( \lambda ) $ and $ f _ {\widetilde \eta \widetilde \eta } ( \lambda ) $ are the spectral densities of $ \eta $ and $ \widetilde \eta $, respectively, and $ f _ {\eta \widetilde \eta } ( \lambda ) $ is their joint spectral density.

References

[1] R. Gallagher, "Information theory and reliable communication" , Wiley (1968)
[2] M.S. Pinsker, "Information and informational stability of random variables and processes" , Holden-Day (1964) (Translated from Russian)
How to Cite This Entry:
Information, transmission rate of. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Information,_transmission_rate_of&oldid=47354
This article was adapted from an original article by R.L. DobrushinV.V. Prelov (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article