Question

Show that if state \(i\) is recurrent and state \(i\) does not communicate with state \(j\), then \(P_{i j}=0 .\) This implies that once a process enters a recurrent class of states it can never leave that class. For this reason, a recurrent class is often referred to as a closed class.

Short answer

Since state i is recurrent and does not communicate with state j, there do not exist positive integers n and m such that \(P_{ij}^{(n)}>0\) and \(P_{ji}^{(m)}>0\). This implies that \(P_{ij}=0\), meaning once a process enters a recurrent state (a closed class), it can never leave that class.

Step-by-step solution

State the Definitions

Before we begin, let's go over few important definitions: 1. Recurrent State: State i is recurrent if and only if the expected number of times it is visited is infinite, i.e., once a process enters a recurrent state, there is a 100% chance that it will return to that state eventually (and infinitely many times). 2. Communicating States: States i and j are said to communicate if there exist positive integers n and m such that the probabilities of transitioning from state i to state j in n steps and from state j to state i in m steps are both strictly positive. 3. Transition Probabilities: \(P_{ij}\) denotes the probability that a process transitions from state i to state j in one step.

Show That Transition Probabilities are Non-Negative

Recall that the transition probability \(P_{ij}\) is the probability that a process transitions from state i to state j. Since probabilities are always non-negative, we have that \(0 \le P_{ij} \le 1\).

Use the Definitions to Derive the Result

Now, we will use the given conditions: state i is recurrent and i does not communicate with state j. 1. Since state i is recurrent, there is a 100% chance that it will return to state i eventually. Mathematically, this means that the sum of probabilities of transitioning from state i to state i in any number of steps is equal to 1. 2. Since state i does not communicate with state j, there do not exist positive integers n and m such that \(P_{ij}^{(n)}>0\) and \(P_{ji}^{(m)}>0\) (where \(P_{ij}^{(n)}\) and \(P_{ji}^{(m)}\) denote the probabilities of transitioning from state i to state j in n steps and from state j to state i in m steps, respectively). In particular, \(P_{ij}^{(1)}=P_{ij}=0\).

Conclude

From step 3, we found that \(P_{ij}=0\), as there is no positive integer n such that the probability of transitioning from state i to state j in n steps is strictly positive. This means that once a process enters a recurrent state, it can never leave the class of recurrent states (since it can never transition to a non-communicating state). Thus, a recurrent class is often referred to as a closed class, and we have proved the given proposition.