Question

Consider a time reversible continuous-time Markov chain having infinitesimal transition rates \(q_{i j}\) and limiting probabilities \(\left\\{P_{i}\right\\} .\) Let \(A\) denote a set of states for this chain, and consider a new continuous-time Markov chain with transition rates \(q_{i j}^{*}\) given by $$ q_{i j}^{*}=\left\\{\begin{array}{ll} c q_{i j}, & \text { if } i \in A, j \notin A \\ q_{i j}, & \text { otherwise } \end{array}\right. $$ where \(c\) is an arbitrary positive number. Show that this chain remains time reversible, and find its limiting probabilities.

Short answer

The new chain remains time-reversible since the detailed balance equations hold for the modified transition rates. The new limiting probabilities, P*_i, have the same functional form as the original probabilities, P_i, and are equal to them (P*_i = P_i).

Step-by-step solution

Define the new transition rates

Define Q* as the new transition rates matrix with elements q_(ij)*, given by: q_(ij)* = c * q_(ij) if i in A and j not in A q_(ij)* = q_(ij) otherwise

Prove time-reversibility for the new chain

For time-reversibility, we have to check if the detailed balance equations are satisfied with the new transition rates. The detailed balance equations are given by: P_i * q_(ij) = P_j * q_(ji) For states i,j such that i is in A and j is not in A, we have q_(ij)* = c * q_(ij), and q_*(ji) = q_(ji). Check the balance equations for these pairs of states: P_i * (c * q_(ij)) = P_j * q_(ji) Since the original chain was time-reversible, we know that P_i * q_(ij) = P_j * q_(ji). Therefore, the detailed balance equations also hold for the new chain with the modified transition rates for states i in A and j not in A. For all other pairs of states, q_(ij)* = q_(ij), and the detailed balance equations directly hold since they held for the original chain. Therefore, the new chain remains time-reversible.

Find the new limiting probabilities

To find the limiting probabilities for the new chain, we need to solve the balance equations. Define P*_i as the new limiting probabilities. The balance equations for the new chain can be written as follows: P*_i * q_(ij)* = P*_j * q_(ji)* For states i,j such that i is in A and j is not in A, we have: P*_i * (c * q_(ij)) = P*_j * q_(ji) and for all other pairs of states, we have: P*_i * q_(ij) = P*_j * q_(ji) We can see that the equations for the new limiting probabilities are essentially the same as for the original chain, with a factor of 'c' for transitions from a state in A to a state not in A. Therefore, the new limiting probabilities P*_i will have the same functional form as the original probabilities P_i, but with a normalization factor due to the modified transition rates. Let the normalization factor be k, then the new limiting probabilities can be written as: P*_i = k * P_i To find k, we need to use the condition that the probabilities must sum up to 1: sum(P*_i) = 1 Substitute P*_i, and simplify: k * sum(P_i) = 1 Since sum(P_i) = 1 for the original chain, we have k = 1. Therefore, the new limiting probabilities are equal to the original probabilities: P*_i = P_i

Key concepts

Transition Rates

In the realm of Markov chains, particularly when discussing continuous-time Markov chains, transition rates are fundamental elements that dictate the dynamics of the system. They are denoted by the symbols like \(q_{ij}\) and represent the instantaneous rate at which the process transitions from state \(i\) to state \(j\). To put it simply, the higher the transition rate \(q_{ij}\), the more frequently the system is likely to move from state \(i\) to state \(j\).

In the exercise, the concept of transition rates is expanded by introducing a new set of rates, \(q_{ij}^*\), which are modified based on whether state \(i\) is within a specified set \(A\). This modification is implemented by multiplying the original rate by a constant \(c\) when transitioning from a state in \(A\) to a state not in \(A\), and keeping it the same otherwise. It's this alteration of rates that the problem then examines, specifically regarding the time reversibility and limiting probabilities of the new chain.

Limiting Probabilities

Limiting probabilities are a cornerstone concept for understanding Markov chains. They are the probabilities to which the system settles as time progresses towards infinity. Written mathematically, if \(P_i\) is the limiting probability of state \(i\), this is the likelihood that the system is in state \(i\) after a long period. Determining limiting probabilities is a matter of solving the balance equations, ensuring that the probabilities correctly reflect the long-term behavior of the system.

In our exercise, the modification of transition rates raises the question whether these changes affect the limiting probabilities. It's somewhat intuitive to think that altering how states communicate would shift their long-term probabilities. However, the beauty of this problem lies in the demonstration that despite the changes in rates between certain states, the limiting probabilities themselves remain unaffected - as long as the process continues to satisfy the detailed balance equations.

Detailed Balance Equations

The focus on detailed balance equations is crucial when dealing with time reversibility in Markov chains. These equations ensure that for every pair of states \(i\) and \(j\), the rate of flow from \(i\) to \(j\) is exactly balanced by the rate of flow from \(j\) to \(i\), given the system's limiting probabilities. The equations are written as \(P_i q_{ij} = P_j q_{ji}\), meaning that the probability of being in state \(i\) and moving to state \(j\) equals the probability of being in state \(j\) and moving to state \(i\).

This concept is pivotal in the provided exercise, as it forms the basis for demonstrating time reversibility in the modified chain. By showing that these equalities hold even after changing the transition rates, we establish that the altered system remains in detailed balance, thus preserving the essential characteristics of a time-reversible Markov chain.

Continuous-Time Markov Chain

A continuous-time Markov chain (CTMC) is an extension of the simple Markov chain to a continuous time domain. Unlike its discrete-time counterpart where transitions occur at specific intervals, a CTMC allows for changes between states at any point in time, guided by the transition rates \(q_{ij}\). This makes CTMCs particularly useful for modeling real-world processes that don't operate on a synchronized clock.

The original exercise illustrates a time reversible CTMC, which is a specific type of CTMC having the property that the system behaves the same way forward in time as it does in reverse, given the detailed balance conditions are satisfied. This time reversibility is an insightful property that simplifies many aspects of the mathematical treatment of such chains, including the analysis of their equilibrium behavior and calculation of limiting probabilities.