Question

Show that the stationary probabilities for the Markov chain having transition probabilities \(P_{i, j}\) are also the stationary probabilities for the Markov chain whose transition probabilities \(Q_{i, j}\) are given by $$ Q_{i, j}=P_{i, j}^{k} $$ for any specified positive integer \(k\).

Short answer

In conclusion, we have shown that the stationary probabilities for a Markov chain with transition probabilities \(P_{i,j}\) are also the stationary probabilities for the Markov chain with transition probabilities \(Q_{i,j} = P_{i,j}^k\), for any specified positive integer \(k\). We achieved this by expanding the equations for the stationary probabilities of both Markov chains and showing that the stationary probabilities of the original Markov chain with matrix \(P\) satisfy the stationary probability conditions for the modified Markov chain having matrix \(Q\) as well.

Step-by-step solution

Define stationary probabilities and Markov chains

A stationary probability distribution (also known as a stationary distribution or steady-state distribution) for a Markov chain is a probability distribution that remains unchanged as the process transitions from one state to another. Consider two Markov chains with transition probability matrices \(P\) and \(Q\), where the components of \(Q\) are obtained by raising the components of \(P\) to the power \(k\) (\(Q_{i,j} = P_{i,j}^k)\). Let \(\pi = (\pi_1, \pi_2, ..., \pi_n)\) denote the stationary probability distribution for the original Markov chain with the transition probability matrix \(P\), where \(n\) is the number of states.

Apply the definition of stationary probabilities

According to the definition of stationary probabilities, \(\pi\) should satisfy the following equation for the Markov chain with the transition probability matrix \(P\): \[ \pi P = \pi \] We need to prove that the same stationary probability distribution \(\pi\) also satisfies this equation for the Markov chain with the transition probability matrix \(Q\): \[ \pi Q = \pi \]

Expand the equation for the stationary probabilities of Q

Substitute \(Q_{i,j}\) with \(P_{i,j}^k\), and expand the equation to obtain: \[ \pi_1 Q_{1,1} + \pi_2 Q_{2,1} + ... + \pi_n Q_{n,1} = \pi_1 \\ \pi_1 Q_{1,2} + \pi_2 Q_{2,2} + ... + \pi_n Q_{n,2} = \pi_2 \\ \cdots \\ \pi_1 Q_{1,n} + \pi_2 Q_{2,n} + ... + \pi_n Q_{n,n} = \pi_n \] Replace the \(Q_{i,j}\) with \(P_{i,j^k}\) in the expanded equations: \[ \pi_1 P_{1,1}^k + \pi_2 P_{2,1}^k + ... + \pi_n P_{n,1}^k = \pi_1 \\ \pi_1 P_{1,2}^k + \pi_2 P_{2,2}^k + ... + \pi_n P_{n,2}^k = \pi_2 \\ \cdots \\ \pi_1 P_{1,n}^k + \pi_2 P_{2,n}^k + ... + \pi_n P_{n,n}^k = \pi_n \]

Verify the stationary probability conditions for Q

We know that the stationary probabilities of P satisfy the following equation: \[ \pi_1 P_{1,1} + \pi_2 P_{2,1} + ... + \pi_n P_{n,1} = \pi_1 \\ \pi_1 P_{1,2} + \pi_2 P_{2,2} + ... + \pi_n P_{n,2} = \pi_2 \\ \cdots \\ \pi_1 P_{1,n} + \pi_2 P_{2,n} + ... + \pi_n P_{n,n} = \pi_n \] Now raise both sides of these equations to the power of \(k\), and we will have: \[ (\pi_1 P_{1,1} + \pi_2 P_{2,1} + ... + \pi_n P_{n,1})^k = \pi_1^k \\ (\pi_1 P_{1,2} + \pi_2 P_{2,2} + ... + \pi_n P_{n,2})^k = \pi_2^k \\ \cdots \\ (\pi_1 P_{1,n} + \pi_2 P_{2,n} + ... + \pi_n P_{n,n})^k = \pi_n^k \] These equations are exactly the same as the ones we obtained in step 3 for the stationary probabilities of \(Q\). This means that the stationary probability distribution \(\pi\) for the Markov chain with the transition probability matrix \(P\) also satisfies the stationary probability conditions for the Markov chain with the transition probability matrix \(Q\).

Conclusion

Hence, the stationary probabilities for a Markov chain with transition probabilities \(P_{i,j}\) are also the stationary probabilities for the Markov chain with transition probabilities \(Q_{i,j} = P_{i,j}^k\), for any specified positive integer \(k\).