Matrix of transition probabilities

The matrix $ P _ {t} = \| p _ {ij} ( t) \| $ of transition probabilities in time $ t $ for a homogeneous Markov chain $ \xi ( t) $ with at most a countable set of states $ S $:

$$ p _ {ij} ( t) = {\mathsf P} \{ \xi ( t) = j \mid \xi ( 0) = i \} ,\ \ i, j \in S. $$

The matrices $ \| p _ {ij} ( t) \| $ of a Markov chain with discrete time or a regular Markov chain with continuous time satisfy the following conditions for any $ t > 0 $ and $ i, j \in S $:

$$ p _ {ij} ( t) \geq 0,\ \ \sum _ {j \in S } p _ {ij} ( t) = 1, $$

i.e. they are stochastic matrices (cf. Stochastic matrix), while for irregular chains

$$ p _ {ij} ( t) \geq 0,\ \ \sum _ {j \in S } p _ {ij} ( t) \leq 1, $$

such matrices are called sub-stochastic.

By virtue of the basic (Chapman–Kolmogorov) property of a homogeneous Markov chain,

$$ p _ {ij} ( s+ t) = \sum _ {k \in S } p _ {ik} ( s) p _ {kj} ( t), $$

the family of matrices $ \{ {P _ {t} } : {t > 0 } \} $ forms a multiplicative semi-group; if the time is discrete, this semi-group is uniquely determined by $ P _ {1} $.

References

[a1]	K.L. Chung, "Elementary probability theory with stochastic processes" , Springer (1974)