View source for Information, transmission rate of

<!--
i0511601.png
$#A+1 = 17 n = 0
$#C+1 = 17 : ~/encyclopedia/old_files/data/I051/I.0501160 Information, transmission rate of
Automatically converted into TeX, above some diagnostics.
Please remove this comment and the {{TEX|auto}} line below,
if TeX found to be correct.
-->

{{TEX|auto}}
{{TEX|done}}

A quantity characterizing the amount of information that is contained in the output signal of a [[Communication channel|communication channel]] relative to the input signal, calculated in a unit of time (cf. [[Information, amount of|Information, amount of]]). If

$$ 
\eta  =  \{ {\eta ( t) } : {- \infty < t < \infty } \}
,\ \ 
\widetilde \eta    =  \{ {\widetilde \eta   ( t) } : {- \infty < t < \infty } \}
$$

are stochastic processes in discrete or continuous time, being the input and output signals of a communication channel, then the quantity

$$ \tag{* }
R  =  \lim\limits _ {T - t \rightarrow \infty } \ 

\frac{1}{T-}
 1
I ( \eta _ {t}  ^ {T} , \widetilde \eta   {} _ {t}  ^ {T} ) ,
$$

is the transmission rate of information (if the limit exists). Here  $  I ( \cdot , \cdot ) $
is the amount of information,  $  \eta _ {t}  ^ {T} = \{ {\eta ( s) } : {t < s \leq  T } \} $
is the segment  $  [ t , T ] $
of  $  \eta $
and  $  \widetilde \eta   {} _ {t}  ^ {T} $
is analogously defined. The existence of the limit in (*) has been proved for the large class of channels in which the signals  $  \eta $
and  $  \widetilde \eta   $
are stationary and stationarily-related stochastic processes. An explicit computation of the transmission rate of information is possible, in particular, for a [[Memoryless channel|memoryless channel]] and a [[Gaussian channel|Gaussian channel]]. E.g., for a Gaussian channel, whose signals  $  \eta $
and  $  \widetilde \eta   $
are Gaussian stationary processes forming a joint Gaussian stationary pair of processes, the transmission rate of information is given by

$$ 
R  =  - 
\frac{1}{2}
 \int\limits _ {- \infty } ^  \infty  
 \mathop{\rm log} \ 
\left (
1 -

\frac{| f _ {\eta \widetilde \eta   }  ( \lambda ) |  ^ {2} }{f _ {\eta \eta }  ( \lambda ) f _ {\widetilde \eta   \widetilde \eta   }  ( \lambda ) }

\right ) \ 
d \lambda ,
$$

where  $  f _ {\eta \eta }  ( \lambda ) $
and  $  f _ {\widetilde \eta   \widetilde \eta   }  ( \lambda ) $
are the spectral densities of  $  \eta $
and  $  \widetilde \eta   $, 
respectively, and  $  f _ {\eta \widetilde \eta   }  ( \lambda ) $
is their joint [[Spectral density|spectral density]].

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  R. Gallagher,   "Information theory and reliable communication" , Wiley  (1968)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  M.S. Pinsker,   "Information and informational stability of random variables and processes" , Holden-Day  (1964)  (Translated from Russian)</TD></TR></table>