Question

Suppose that three boys A, B, and C are throwing a ball from one to another. Whenever A has the ball, he throws it to B with a probability of 0.2 and to C with a probability of 0.8. Whenever B has the ball, he throws it to A with a probability of 0.6 and to C with a probability of 0.4. Whenever C has the ball, he is equally likely to throw it to either A or B.

a. Consider this process to be a Markov chain and construct the transition matrix.

b. If each of the three boys is equally likely to have the ball at a certain time n, which boy is most likely to have the ball at time\(n + 2\).

Short answer

1. The transition matrix is given as:\(p = \left[ {\begin{array}{*{20}{c}}0&{0.2}&{0.8}\\{0.6}&0&{0.4}\\{0.5}&{0.5}&0\end{array}} \right]\)
2. boy C is most likely to have the ball at time\(n + 2\)with probability0.386

Step-by-step solution

Explaining stochastic process.

Stochastic process can be defined as the sequence of the random variable

\({X_1}...{X_i}\)noted against discrete time parameter.

Here\({X_i}\)is the i^th time period. In stochastic process the value \({X_i}\)depends on the preceding values observed at times \(t - 1,t - 2...,1\).

constructing the transition probability matrix.

A ball throwing game between three boys A, B and C. A throws ball to B with probability 0.2 and to C with probability 0.8 throws ball to A with probability 0.6

and to C with probability 0.4.finally C throws the ball to A and to B with equal probability that is 0.5

Let’s consider that the game forms a markov chain. So, the transition probabilities of the game can be given as

	A	B	C
A	0	0.2	0.8
B	0.6	0	0.4
C	0.5	0.5	0

Hence the transition matrix is given as:

\(p = \left[ {\begin{array}{*{20}{c}}0&{0.2}&{0.8}\\{0.6}&0&{0.4}\\{0.5}&{0.5}&0\end{array}} \right]\)

calculating which boy is likely to have the ball at time \(n + 2\)

Each of the three boys is equally likely to have the ball at a certain time n.

Find out the boy who is most likely to have the ball at time \(n + 2\)

Lets find the seconds step transition matrix.

\({p^{\left( 2 \right)}} = p \times p\)

\(\begin{aligned}{}{p^{\left( 2 \right)}} &= \left[ {\begin{aligned}{}0&{0.2}&{0.8}\\{0.6}&0&{0.4}\\{0.5}&{0.5}&0\end{aligned}} \right] \times \left[ {\begin{aligned}{}0&{0.2}&{0.8}\\{0.6}&0&{0.4}\\{0.5}&{0.5}&0\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{}{0.52}&{0.4}&{0.08}\\{0.2}&{0.32}&{0.48}\\{0.3}&{0.1}&{0.6}\end{aligned}} \right]\end{aligned}\)

Defining a vector \(v = \left[ {\begin{aligned}{}{\frac{1}{3}}&{\frac{1}{3}}&{\frac{1}{3}}\end{aligned}} \right]\)

That represents the probability of holding the ball at time n by each of the boys. So, the probability of holding the ball at time \(n + 2\) can be obtained by multiplying v by \({p^{\left( 2 \right)}}\).

\(\begin{aligned}{}v{p^{\left( 2 \right)}} &= \left[ {\begin{aligned}{}{\frac{1}{3}}&{\frac{1}{3}}&{\frac{1}{3}}\end{aligned}} \right] \times \left[ {\begin{aligned}{}{0.52}&{0.4}&{0.08}\\{0.2}&{0.32}&{0.48}\\{0.3}&{0.1}&{0.6}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{}{0.34}&{0.273}&{0.386}\end{aligned}} \right]\end{aligned}\)

It can be observed that each of the three boys is equally likely to have the ball at a certain time n, boy C is most likely with probability 0.386 to have the ball at time\(\left( {n + 2} \right)\).