Question

(a) Show that the limiting probabilities of the reversed Markov chain are the same as for the forward chain by showing that they satisfy the equations $$ \pi_{j}=\sum_{i} \pi_{i} Q_{i j} $$ (b) Give an intuitive explanation for the result of part (a).

Short answer

In summary, the limiting probabilities of the reversed Markov chain are the same as those for the forward chain because they both satisfy the equation \(\pi_j = \sum_{i} \pi_i Q_{ij}\). Intuitively, this is because the long-term behavior of the Markov chain is not affected by the direction in which it is traversed, and the limiting probabilities represent the proportion of time spent in each state, which is independent of the direction of the chain.

Step-by-step solution

Understanding the Forward Markov Chain

For a (discrete-time) forward Markov chain with state space \(S\), we have a transition probability matrix \(P\), where the element \(P_{ij}\) is the probability of transitioning from state \(i\) to state \(j\). The limiting probabilities \(\pi = (\pi_i)_{i\in S}\) are the long-term probabilities that the chain is in each state \(i\), known as the stationary distribution, which satisfy \(\pi_i \geq 0\) for all \(i\), \(\sum_{i} \pi_i = 1\), and \[\pi = \pi P .\]

Understanding the Reversed Markov Chain

For a reversed Markov chain, we still have the state space \(S\) and stationary distribution \(\pi\). We define the transition probability matrix \(Q\) for the reversed chain by: \[Q_{ij} = \frac{\pi_j}{\pi_i} P_{ji},\] which represents the probability of transitioning from state \(j\) to state \(i\), reversing the direction of the forward chain.

Show that the Limiting Probabilities Satisfy the Equations

Now we need to show that the limiting probabilities of the reversed Markov chain, represented by the vector \(\pi\), satisfy the given equation: \[\pi_j = \sum_{i} \pi_i Q_{ij} .\] Let's compute the right-hand side of this equation and show that it indeed equals \(\pi_j\): \[\sum_{i} \pi_i Q_{ij} = \sum_{i} \pi_i \left(\frac{\pi_j}{\pi_i}P_{ji}\right) = \pi_j \sum_{i} P_{ji} .\] Recall that each row in the transition probability matrix \(P\) must sum to 1, so we get \[\pi_j \sum_{i} P_{ji} = \pi_j (1) = \pi_j .\] Thus, the limiting probabilities of the reversed Markov chain are the same as those for the forward chain.

Intuitive Explanation

The reversed Markov chain is essentially "playing the forward Markov chain backward". Since the stationary distribution reflects long-term behavior, it is reasonable that the limiting probabilities should be the same for both the forward and reversed chains. At a high level, this is because the long-term behavior of the chain is not affected by the direction in which it is traversed. The limiting probabilities represent the proportion of time spent in each state over the long run, and these proportions are independent of the direction of the chain.