Channel with a finite memory

A communication channel for which the statistical properties of the output signal at a time $ t $ are determined by the input signals transmitted at the times $ t ^ \prime $, $ t - m \leq t ^ \prime \leq t $( and therefore do not depend on the signals transmitted prior to the time $ t - m $); the number $ m $ is called the size (or length) of the memory of the channel.

More precisely, a discrete-time communication channel where the input and output signals are given, respectively, by random sequences $ \eta = ( . . . , \eta _ {-} 1 , \eta _ {0} , \eta _ {1} , . . . ) $ and $ \widetilde \eta = ( \widetilde \eta _ {-} 1 , \widetilde \eta _ {0} , \widetilde \eta _ {1} , . . . ) $ with values in the spaces $ ( Y, S _ {Y} ) $ and $ ( \widetilde{Y} , S _ {\widetilde{Y} } ) $ is called a channel with a finite memory if a compatible set of conditional distributions

$$ {\mathsf P} \{ \widetilde \eta {} _ {k} ^ {n} \in \widetilde{A} \mid \ \eta _ {- \infty } ^ {n} \} , $$

by means of which such a channel can be defined, satisfies for any $ i, j, k, n $, and $ \widetilde{A} , \widetilde{B} $ the conditions

$$ {\mathsf P} \{ \widetilde \eta {} _ {k} ^ {n} \in \widetilde{A} \mid \ \eta _ {- \infty } ^ {n} \} = {\mathsf P} \{ \widetilde \eta {} _ {k} ^ {n} \in \widetilde{A} \mid \ \eta _ {k - m } ^ {n} \} , $$

$$ {\mathsf P} \{ \widetilde \eta {} _ {i} ^ {j} \in \widetilde{B} , \widetilde \eta {} _ {k} ^ {n} \in \widetilde{A} \mid \eta _ {- \infty } ^ {n} \} = $$

$$ = \ {\mathsf P} \{ \widetilde \eta {} _ {i} ^ {j} \in \widetilde{B} \mid \eta _ {- \infty } ^ {n} \} \cdot {\mathsf P} \{ \widetilde \eta {} _ {k} ^ {n} \in \widetilde{A} \mid \eta _ {- \infty } ^ {n} \} . $$

Here $ \widetilde \eta _ {k} ^ {n} = ( \widetilde \eta _ {k} \dots \widetilde \eta _ {n} ) $, $ \eta _ {- \infty } ^ {n} = ( . . . , \eta _ {n - 1 } , \eta _ {n} ) $, and $ \widetilde{A} $( respectively, $ \widetilde{B} $) is a set in the direct product of $ n - k + 1 $( respectively, $ j - i + 1 $) copies of $ ( \widetilde{Y} , S _ {\widetilde{Y} } ) $. A continuous-time channel with a finite memory is defined similarly.

References

[1]	A.Ya. Khinchin, "On the basic theorems of information theory" Uspekhi Mat. Nauk , 11 : 1 (1956) pp. 17–75 (In Russian)
[2]	A.A. Feinstein, "Foundations of information theory" , McGraw-Hill (1968)
[3]	J. Wolfowitz, "Coding theorems of information theory" , Springer (1964)