Absorbing state
of a Markov chain $ \xi (t) $
2020 Mathematics Subject Classification: Primary: 60J10 [MSN][ZBL]
A state $ i $ such that
$$ {\mathsf P} \{ \xi (t) = i \mid \xi (s) = i \} = 1 \ \textrm{ for } \textrm{ any } t \geq s. $$
An example of a Markov chain with absorbing state $ 0 $ is a branching process.
The introduction of additional absorbing states is a convenient technique that enables one to examine the properties of trajectories of a Markov chain that are associated with hitting some set.
Example. Consider the set $ S $ of states of a homogeneous Markov chain $ \xi (t) $ with discrete time and transition probabilities
$$ p _ {ij} = {\mathsf P} \{ \xi (t+1) = j \mid \xi (t) = i \} , $$
in which a subset $ H $ is distinguished and suppose one has to find the probabilities
$$ q _ {ih} = {\mathsf P} \{ \xi ( \tau (H)) = h \mid \xi (0) = i \} , \ i \in S,\ h \in H, $$
where $ \tau (H) = \mathop{\rm min} \{ {t > 0 } : {\tau (t) \in H } \} $ is the moment of first hitting the set $ H $. If one introduces the auxiliary Markov chain $ \xi ^ {*} (t) $ differing from $ \xi (t) $ only in that all states $ h \in H $ are absorbing in $ \xi ^ {*} (t) $, then for $ h \in H $ the probabilities
$$ p _ {ih} ^ {*} (t) = {\mathsf P} \{ \xi ^ {*} (t) = h \mid \xi ^ {*} (0) = i \} = $$
$$ = \ {\mathsf P} \{ \tau (H) \leq t, \xi ( \tau (H)) = h \mid \xi (0) = i \} $$
are monotonically non-decreasing for $ t \uparrow \infty $ and
$$ \tag{* } q _ {ih} = \lim\limits _ {t \rightarrow \infty } p _ {ih} ^ {*} (t), \ i \in S,\ h \in H. $$
By virtue of the basic definition of a Markov chain
$$ p _ {ih} ^ {*} (t + 1) = \ \sum _ {j \in S } p _ {ij} p _ {ih} ^ {*} (t), \ t \geq 0,\ i \in S \setminus H,\ h \in H, $$
$$ p _ {hh} ^ {*} (t) = 1,\ h \in H; \ p _ {ih} ^ {*} (t) = 0,\ i, h \in H, i \neq h. $$
The passage to the limit for $ t \rightarrow \infty $ taking into account (*) gives a system of linear equations for $ q _ {ih} $:
$$ q _ {ih} = \sum _ {j \in S } p _ {ij} q _ {ih} ,\ \ i \in S \setminus H,\ h \in H, $$
$$ q _ {hh} = 1,\ h \in H ; \ q _ {ih} = 0,\ i, h \in H, i \neq h. $$
References
[F] | W. Feller, "An introduction to probability theory and its applications", 1, Wiley (1968) |
Absorbing state. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Absorbing_state&oldid=45007