Question

Suppose that a Markov chain has four states 1, 2, 3, 4, and stationary transition probabilities as specified by the following transition matrix

\(p = \left[ {\begin{array}{*{20}{c}}{\frac{1}{4}}&{\frac{1}{4}}&0&{\frac{1}{2}}\\0&1&0&0\\{\frac{1}{2}}&0&{\frac{1}{2}}&0\\{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}\end{array}} \right]\):

a.If the chain is in state 3 at a given timen, what is the probability that it will be in state 2 at timen+2?

b.If the chain is in state 1 at a given timen, what is the probability it will be in state 3 at timen+3?

Short answer

1. The probability that it will be in state-2 at a time\(n + 2\) is \(\left[ {\frac{1}{8}} \right]\).
2. The probability that it will be in state-3 at a time \(n + 3\) is \(\left[ {\frac{1}{8}} \right]\) .

Step-by-step solution

Given Information

Given that the Markov chain has four states \(1,2,3,4\). Given transition matrix to find state-2 at time \(n + 2\) and state-3 at time \(n + 3\).

Define the Markov chain for this problem

Here we calculate the Markov chain for particular rows and columns. Not calculate the whole transition matrix.

(a) Calculate the chain in state-2 at the time \(n + 2\)

Here, wee define the square matrix forthef third row and a second column for the given transition probability matrix. Then from the given matrix to find that

\(\begin{aligned}{}{p^2} &= p \times p\\ &= \left[ {\begin{aligned}{{}{}}{\frac{1}{4}}&{\frac{1}{4}}&0&{\frac{1}{2}}\\0&1&0&0\\{\frac{1}{2}}&0&{\frac{1}{2}}&0\\{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}\end{aligned}} \right] \times \left[ {\begin{aligned}{{}{}}{\frac{1}{4}}&{\frac{1}{4}}&0&{\frac{1}{2}}\\0&1&0&0\\{\frac{1}{2}}&0&{\frac{1}{2}}&0\\{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}\end{aligned}} \right]\\ &= \left[ {\frac{1}{8}} \right]\end{aligned}\)

Finally it is solved this way.

(b) Calculate the chain in state-3 at time \(n + 3\)

Here we calculate the cube of the matrix of first row and third column of the given transition matrix. Then it is defined as

\(\begin{aligned}{}{p^3} &= {p^2} \times p\\ &= \left[ {\begin{aligned}{*{20}{c}}{\frac{1}{4}}&{\frac{1}{4}}&0&{\frac{1}{2}}\\0&1&0&0\\{\frac{1}{2}}&0&{\frac{1}{2}}&0\\{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}\end{aligned}} \right] \times \left[ {\begin{aligned}{*{20}{c}}{\frac{1}{4}}&{\frac{1}{4}}&0&{\frac{1}{2}}\\0&1&0&0\\{\frac{1}{2}}&0&{\frac{1}{2}}&0\\{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}&{\frac{1}{4}}\end{aligned}} \right]\\ &= \left[ {\frac{1}{8}} \right]\end{aligned}\)

Hence it is solved by this process.