Question

Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state \(i, i=0,1,2,3\), if the first urn contains \(i\) white balls. At each step, we draw one ball from each urn and place the ball drawn from the first urn into the second, and conversely with the ball from the second urn. Let \(X_{n}\) denote the state of the system after the \(n\) th step. Explain why \(\left\\{X_{n}, n=0,1,2, \ldots\right\\}\) is a Markov chain and calculate its transition probability matrix.

Short answer

The sequence { \(X_n\), n=0,1,2,...} is a Markov chain because the probability of each transition depends only on the current state of the system. The transition probability matrix P for the Markov chain is: \[ P = \begin{pmatrix} 0 & 1 & 0 & 0 \\ \frac{2}{9} & 0 & \frac{2}{9} & 0 \\ 0 & \frac{2}{9} & 0 & \frac{2}{9} \\ 0 & 0 & 1 & 0 \end{pmatrix} \] This matrix shows the probabilities of transitioning from one state to another in the given system of two urns with white and black balls.

Step-by-step solution

First, let's identify the possible transitions between states i. - If the system is in state 0: urn 1 has 0 white balls and urn 2 has 3 white balls. In this state, only one transition is possible: one white ball is drawn from urn 2 and placed in urn 1, and the system moves to state 1. - If the system is in state 1: urn 1 has 1 white ball and urn 2 has 2 white balls. There are two possible transitions: either both urns exchange a white ball and the system moves back to state 0, or both urns exchange black balls and the system moves to state 2. - If the system is in state 2: urn 1 has 2 white balls and urn 2 has 1 white ball. There are two possible transitions, similar to state 1: either both urns exchange white balls and the system moves to state 1, or both urns exchange black balls and the system moves to state 3. - If the system is in state 3: urn 1 has 3 white balls and urn 2 has 0 white balls, similar to state 0, only one transition is possible: 1 white ball is drawn from urn 1 and placed in urn 2, and the system moves to state 2. #Step 2: Verify if the system is a Markov chain#

Since the probability of each transition depends only on the current state of the system, the sequence { \(X_n\), n=0,1,2,...} is a Markov chain. #Step 3: Calculate transition probabilities#

Now we will calculate the transition probabilities for each possible transition. - From state 0 to state 1: Since there is only 1 white ball in urn 2 and all balls are different, the probability of this transition is 1, i.e., \(P_{01} = 1\). - From state 1 to state 0: The probability of picking a white ball from both urns is \(\frac{1}{3}\times\frac{2}{3} = \frac{2}{9}\), i.e., \(P_{10} = \frac{2}{9}\). - From state 1 to state 2: The probability of picking a black ball from both urns is \(\frac{2}{3}\times\frac{1}{3} = \frac{2}{9}\), i.e., \(P_{12} = \frac{2}{9}\). - From state 2 to state 1: Similar to state 1 to state 0, the probability is \(P_{21} = \frac{2}{9}\). - From state 2 to state 3: Similar to state 1 to state 2, the probability is \(P_{23} = \frac{2}{9}\). - From state 3 to state 2: Since there is only 1 white ball in urn 1 and all balls are different, the probability is 1, i.e., \(P_{32} = 1\). #Step 4: Form the transition probability matrix#

Using the transition probabilities calculated in the previous step, we can now form the transition probability matrix P for the Markov chain. The matrix will have dimensions 4x4 (since there are 4 possible states). \[ P = \begin{pmatrix} 0 & 1 & 0 & 0 \\ \frac{2}{9} & 0 & \frac{2}{9} & 0 \\ 0 & \frac{2}{9} & 0 & \frac{2}{9} \\ 0 & 0 & 1 & 0 \end{pmatrix} \] This matrix represents the Markov chain describing the system, where each element \(P_{ij}\) represents the probability of transitioning from state i to state j.