Absorbing state

of a Markov chain $ \xi (t) $

2020 Mathematics Subject Classification: Primary: 60J10 [MSN][ZBL]

A state $ i $ such that

$$ {\mathsf P} \{ \xi (t) = i \mid \xi (s) = i \} = 1 \ \textrm{ for } \textrm{ any } t \geq s. $$

An example of a Markov chain with absorbing state $ 0 $ is a branching process.

The introduction of additional absorbing states is a convenient technique that enables one to examine the properties of trajectories of a Markov chain that are associated with hitting some set.

Example. Consider the set $ S $ of states of a homogeneous Markov chain $ \xi (t) $ with discrete time and transition probabilities

$$ p _ {ij} = {\mathsf P} \{ \xi (t+1) = j \mid \xi (t) = i \} , $$

in which a subset $ H $ is distinguished and suppose one has to find the probabilities

$$ q _ {ih} = {\mathsf P} \{ \xi ( \tau (H)) = h \mid \xi (0) = i \} , \ i \in S,\ h \in H, $$

where $ \tau (H) = \mathop{\rm min} \{ {t > 0 } : {\tau (t) \in H } \} $ is the moment of first hitting the set $ H $. If one introduces the auxiliary Markov chain $ \xi ^ {*} (t) $ differing from $ \xi (t) $ only in that all states $ h \in H $ are absorbing in $ \xi ^ {*} (t) $, then for $ h \in H $ the probabilities

$$ p _ {ih} ^ {*} (t) = {\mathsf P} \{ \xi ^ {*} (t) = h \mid \xi ^ {*} (0) = i \} = $$

$$ = \ {\mathsf P} \{ \tau (H) \leq t, \xi ( \tau (H)) = h \mid \xi (0) = i \} $$

are monotonically non-decreasing for $ t \uparrow \infty $ and

$$ \tag{* } q _ {ih} = \lim\limits _ {t \rightarrow \infty } p _ {ih} ^ {*} (t), \ i \in S,\ h \in H. $$

By virtue of the basic definition of a Markov chain

$$ p _ {ih} ^ {*} (t + 1) = \ \sum _ {j \in S } p _ {ij} p _ {ih} ^ {*} (t), \ t \geq 0,\ i \in S \setminus H,\ h \in H, $$

$$ p _ {hh} ^ {*} (t) = 1,\ h \in H; \ p _ {ih} ^ {*} (t) = 0,\ i, h \in H, i \neq h. $$

The passage to the limit for $ t \rightarrow \infty $ taking into account (*) gives a system of linear equations for $ q _ {ih} $:

$$ q _ {ih} = \sum _ {j \in S } p _ {ij} q _ {ih} ,\ \ i \in S \setminus H,\ h \in H, $$

$$ q _ {hh} = 1,\ h \in H ; \ q _ {ih} = 0,\ i, h \in H, i \neq h. $$

References

[F]	W. Feller, "An introduction to probability theory and its applications", 1, Wiley (1968)