Question

Consider a process \(\left\\{X_{n}, n=0,1, \ldots\right\\}\), which takes on the values 0,1 , or 2 . Suppose $$ \begin{aligned} &P\left\\{X_{n+1}=j \mid X_{n}=i, X_{n-1}=i_{n-1}, \ldots, X_{0}=i_{0}\right\\} \\ &\quad=\left\\{\begin{array}{ll} P_{i j}^{\mathrm{I}}, & \text { when } n \text { is even } \\ P_{i i}^{\mathrm{II}}, & \text { when } n \text { is odd } \end{array}\right. \end{aligned} $$ where \(\sum_{j=0}^{2} P_{i j}^{\mathrm{I}}=\sum_{j=0}^{2} P_{i j}^{\mathrm{II}}=1, i=0,1,2 .\) Is \(\left\\{X_{n}, n \geqslant 0\right\\}\) a Markov chain? If not, then show how, by enlarging the state space, we may transform it into a Markov chain.

Short answer

The given process \(X_n\) is not a Markov chain, as it does not satisfy the Markov property due to its dependency on the parity of n (odd or even). To transform it into a Markov chain, we can introduce a new process \(Y_n = \{ (X_n, p_n ) \}\) that includes an additional element for the parity of n. By enlarging the state space to \(Y_n\) and defining its probability transition accordingly, we can satisfy the Markov property and transform the process into a Markov chain.

Step-by-step solution

Check if the process satisfies the Markov property

To check if the given process satisfies the Markov property, we need to test if the conditional probability of the next state depends only on the current state and not on past states. Given the conditional probability, we have: \(P\left[X_{n+1}=j \mid X_{n}=i_{0} X_{n-1}=I_{n-1}, \ldots, X_{0}=i_{0}\right]\) However, this conditional probability depends on whether n is even or odd, which means it is indirectly dependent on past states. Therefore, the process does not satisfy the Markov property, and it is not a Markov chain.

Enlarge the state space

To enlarge the state space and transform the process into a Markov chain, we can add an additional element to the state description that keeps track of the parity of n (whether n is odd or even). We can introduce a new process \(Y_n\) defined as: \(Y_n = \{ (X_n, p_n ) \}\), where p_n = 0 when n is even, and p_n = 1 when n is odd. The new state space for the process \(Y_n\) will have 6 possible states: (0,0), (0,1), (1,0), (1,1), (2,0), and (2,1).

Define the probability transition for the new process\(Y_n\)

For the new process \( Y_n \), we can define the probability transition as follows: \(P\left[Y_{n+1}=(j,p_{n+1}) \mid Y_{n}=(i_p, p_n)\right]=\left\\{\begin{array}{ll} P_{U}^{1}, & \text { when } p_n = 0 \\\ P_{U}^{i_p 1}, & \text { when } p_n = 1 \end{array}\right.\) Now, we find that the conditional probability of the next state of the process \( Y_n \) depends only on the current state and not on past states, which satisfies the Markov property.

Conclusion

The original process \(X_n\) is not a Markov chain, as it does not satisfy the Markov property. By enlarging the state space and introducing the process \(Y_n\), which also accounts for the parity of n (odd or even), we have transformed it into a Markov chain.