Question

Suppose that 3 white and 3 black balls are distributed in two urns in such a way that each urn contains 3 balls. We say that the system is in state i if the first urn contains i white balls, i = 0, 1, 2, 3. At each stage, 1 ball is drawn from each urn and the ball drawn from the first urn is placed in the second, and conversely with the ball from the second urn. Let Xn denote the state of the system after the nth stage, and compute the transition probabilities of the Markov chain {Xn, n Ú 0}.

Short answer

The transition probabilities of the Markov chain $\left\{X_n,n\ge 0\right\}$is

$P=\left[\begin{array}{cccc}0& 1& 0& 0\\ \frac{1}{9}& \frac{4}{9}& \frac{4}{9}& 0\\ 0& \frac{4}{9}& \frac{4}{9}& \frac{1}{9}\\ 0& 0& 1& 0\end{array}\right]$

Step-by-step solution

Given Information

We have given that there are 3 white and 3 black balls are distributed in two urns in such a way that each urn contains 3 balls.

Explanation

Let us cosider we have $0$white balls in the first urn. Looking with probability $1$there will be one white ball after a draw because of all balls in the second urn are white. Similarly, if we have 3 white balls, with probability $1$we will have $2$white balls after draw because of all balls in the second urn are black.

Now, consider that there is one white ball in the first urn. After the draw we can have $0,1$or $2$white balls in the first urn. We will have zero white balls if and only if we draw a white ball from the first urn and a black ball from the second urn. The probability for that event is$\frac{1}{3}·\frac{1}{3}$. Then further, we will have one white ball if and only if we draw balls that has the same color. So, the probability for that event is $\frac{1}{3}·\frac{2}{3}+\frac{2}{3}·\frac{1}{3}=\frac{4}{9}$. Finally, we will have two white balls if and only if we draw the black one from the first urn and white one from the second urn, so the probability will be $\frac{2}{3}·\frac{2}{3}=\frac{4}{9}$.

And now, let us consider that there are two white balls in the first urn. After the draw we can have $1,2$or $3$white balls in the first urn. We will have one white balls if and only if we draw white ball from the first urn and a black ball from the second urn. The probability for the event is $\frac{2}{3}·\frac{2}{3}$. And further, we will have two white balls if and only if we draw balls that have the same color. Then, the probability for that event will be $\frac{2}{3}·\frac{1}{3}+\frac{1}{3}·\frac{2}{3}=\frac{4}{9}$. At last, we will have two balls if and only if we draw the black one from the first urn and white one from the second urn. Then the probability will be$\frac{1}{3}·\frac{1}{3}=\frac{1}{9}$.
So, the transition matrix will be represented as

role="math" localid="1648104534442" $P=\left[\begin{array}{cccc}0& 1& 0& 0\\ \frac{1}{9}& \frac{4}{9}& \frac{4}{9}& 0\\ 0& \frac{4}{9}& \frac{4}{9}& \frac{1}{9}\\ 0& 0& 1& 0\end{array}\right]$