Question

Consider the Markov chain in Example 3.10.2 with initial

probability vector \(v = \left( {\frac{1}{2},\frac{1}{2}} \right)\) Where \(p = \left[ {\begin{array}{*{20}{c}}{\frac{1}{3}}&{\frac{2}{3}}\\{\frac{1}{3}}&{\frac{1}{3}}\end{array}} \right]\)

a.Find the probability vector specifying the probabilities

of the states at timen=2.

b.Find the two-step transition matrix

Short answer

1. The probabilities of the states at a time\(n = 2\)is

\(\left[ {\begin{array}{{}{}}{\frac{1}{6}}&{\frac{4}{9}}\\{\frac{1}{9}}&{\frac{1}{6}}\end{array}} \right]\)

2. The two-step transition probability matrix is

\(\left[ {\begin{array}{{}{}}{\frac{1}{3}}&{\frac{4}{9}}\\{\frac{2}{9}}&{\frac{1}{3}}\end{array}} \right]\)

Step-by-step solution

Given Information

The initial probability vector is \(v = \left( {\frac{1}{2},\frac{1}{2}} \right)\) , and the probability matrix is \(p = \left[ {\begin{array}{{}{}}{\frac{1}{3}}&{\frac{2}{3}}\\{\frac{1}{3}}&{\frac{1}{3}}\end{array}} \right]\) .

State the vector and matrix

For this case, we first calculate the two-step transition probability matrix and then calculate for states at the time \(n = 2\) .

Find the probability vector of the states at the time \(n = 2\) 

1. So firstly, calculate the two-step transition probability matrix from the given probability matrix. Then the two-step transition probability matrix is given by

\(\begin{aligned}{}{p^2} &= p \times p\\ &= \left[ {\begin{aligned}{{}{}}{\frac{1}{3}}&{\frac{2}{3}}\\{\frac{1}{3}}&{\frac{1}{3}}\end{aligned}} \right] \times \left[ {\begin{aligned}{{}{}}{\frac{1}{3}}&{\frac{2}{3}}\\{\frac{1}{3}}&{\frac{1}{3}}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{{}{}}{\frac{1}{3}}&{\frac{4}{9}}\\{\frac{2}{9}}&{\frac{1}{3}}\end{aligned}} \right]\end{aligned}\)

Therefore, to calculate the states at the time, \(n = 2\) we have to multiply the two-step transition probability matrix with the initial vector. Then the state is defined as \(\begin{aligned}{}v \times {p^2} &= \left[ {\frac{1}{2},\frac{1}{2}} \right] \times \left[ {\begin{aligned}{{}{}}{\frac{1}{3}}&{\frac{4}{9}}\\{\frac{2}{9}}&{\frac{1}{3}}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{{}{}}{\frac{1}{6}}&{\frac{4}{9}}\\{\frac{1}{9}}&{\frac{1}{6}}\end{aligned}} \right]\end{aligned}\)

Compute the two-step transition probability matrix

For the two-step transition probability matrix, the matrix will be defined as

\(\begin{array}{p^{\left( 2 \right)}} = {p^{\left( 1 \right)}} \times {p^{\left( 1 \right)}}\\ = \left( {\begin{array}{}{\frac{1}{3}}&{\frac{2}{3}}\\{\frac{1}{3}}&{\frac{1}{3}}\end{array}} \right) \times \left( {\begin{array}{}{\frac{1}{3}}&{\frac{2}{3}}\\{\frac{1}{3}}&{\frac{1}{3}}\end{array}} \right)\\ = \left( {\begin{array}{}{\frac{1}{3}}&{\frac{4}{9}}\\{\frac{2}{9}}&{\frac{1}{3}}\end{array}} \right)\end{array}\)