Question

The unique stationary distribution in Exercise 9 is \({\bf{v = }}\left( {{\bf{0,1,0,0}}} \right)\). This is an instance of the following general result: Suppose that a Markov chain has exactly one absorbing state. Suppose further that, for each non-absorbing state \({\bf{k}}\), there is \({\bf{n}}\) such that the probability is positive of moving from state \({\bf{k}}\) to the absorbing state in \({\bf{n}}\) steps. Then the unique stationary distribution has probability 1 in the absorbing state. Prove this result.

Short answer

Proved.

Step-by-step solution

Given information

Referring to exercise 9, the unique stationery distribution is \(v = \left( {0,1,0,0} \right)\),

Proving part

It is to be known that, the 2-step transition matrix that it is possible to get from every non-absorbing state into each of the absorbing states in two steps.

So, no matter what the non-absorbing state is the probability is one that will eventually end up in one absorbing state.

No distribution with a positive probability on any non-absorbing state can be stationary distribution.

All probability in non-absorbing states eventually moves into the absorbing states after sufficiently many transitions.

Hence, proved.