Question

Let $$ p\left(x_{1}, x_{2}\right)=\left(\begin{array}{l} x_{1} \\ x_{2} \end{array}\right)\left(\frac{1}{2}\right)^{x_{1}}\left(\frac{x_{1}}{15}\right), \begin{aligned} &x_{2}=0,1, \ldots, x_{1} \\ &x_{1}=1,2,3,4,5 \end{aligned} $$ zero elsewhere, be the joint pmf of \(X_{1}\) and \(X_{2}\). Determine (a) \(E\left(X_{2}\right)\). (b) \(u\left(x_{1}\right)=E\left(X_{2} \mid x_{1}\right)\). (c) \(E\left[u\left(X_{1}\right)\right]\). Compare the answers of parts (a) and (c). Hint: Note that \(E\left(X_{2}\right)=\sum_{x_{1}=1}^{5} \sum_{x_{2}=0}^{x_{1}} x_{2} p\left(x_{1}, x_{2}\right)\)

Short answer

The results will be specific numbers, which should be the same for parts (a) and (c), showing an application of the law of total expectation: \(E[X_{2}] = E[E[X_{2}|X_{1}]]\).

Step-by-step solution

Compute \(E(X_{2})\)

According to the given pmf and hint, we can find \(E(X_{2})\) by the formula \[E(X_{2}) = \sum_{x_{1}=1}^{5} \sum_{x_{2}=0}^{x_{1}} x_{2} p(x_{1}, x_{2})\],where \(p(x_{1}, x_{2}) = x_{1} (\frac{1}{2})^{x_{1}}(\frac{x_{1}}{15})\) for \(x_{2} = 0,1,\ldots,x_{1}\) and \(x_{1} = 1,2,3,4,5\).

Compute \(E(X_{2}|x_{1})\)

The conditional expectation \(E(X_{2}|x_{1})\) is given by\[E(X_{2}|x_{1}) = \sum_{x_{2}=0}^{x_{1}} x_{2} p(x_{2}|x_{1})\],where \(p(x_{2}|x_{1})\) is the conditional pmf of \(X_{2}\) given \(x_{1}\), and it is\[p(x_{2}|x_{1}) = \frac{p(x_{1}, x_{2})}{p(x_{1} = x_1)}\].Note that \(p(x_{1} = x_1)\) is the marginal pmf of \(x_{1}\), and can be computed as \(p(x_{1} = x_1) = \sum_{x_{2}=0}^{x_{1}} p(x_{1}, x_{2})\).

Compute \(E[E(X_{2}|X_{1})]\)

The expected value of the conditional expectation \(E[X_{2}|X_{1}]\) is\[E[E(X_{2}|X_{1})] = \sum_{x_{1}=1}^{5} E(X_{2}|x_{1}) p(x_{1} = x_1)\],where \(E(X_{2}|x_{1})\) is computed as in step 2 and \(p(x_{1} = x_1)\) is the marginal pmf of \(x_{1}\). By the law of total expectation, this should yield the same result as in step 1.

Key concepts

Expected Value

The concept of expected value is crucial in understanding probability distributions. It provides a measure of the 'central tendency' or average outcome of a random variable. In essence, it’s the long-run average value of repetitions of the experiment it represents.

For a discrete random variable, such as in our exercise, the expected value, denoted as \(E(X)\), is calculated by summing the product of each possible value the random variable can take and the probability of that value occurring:
\[E(X) = \sum_{\text{all } x} x \times p(x)\].

In the context of the exercise, \(E(X_2)\) is found by summing over all values of \(X_1\) and \(X_2\), where each term is a product of \(X_2\)’s value and its joint probability with \(X_1\). This process produces the average value of \(X_2\) when repeated over a long period.

Conditional Probability

Conditional probability is the probability of an event occurring given that another event has already occurred. It’s notated as \(P(A|B)\) and is read as 'the probability of A given B'.

When dealing with joint probability functions, such as \(p(x_{1}, x_{2})\) in our exercise, you can find the conditional probability by dividing the joint probability by the marginal probability of the given condition:
\[P(X_2|x_1) = \frac{p(x_{1}, x_{2})}{p(x_{1})}\].

This effectively updates our beliefs about the likelihood of \(X_2\) in the context of specific values of \(X_1\). Computing the conditional probability is a critical step in finding the conditional expected value \(E(X_2|x_1)\).

Law of Total Expectation

The law of total expectation bridges the gap between marginal and conditional expectations. This law asserts that the expected value of a random variable is equal to the expected value of its expected values given another variable. Formally stated:
\[E(X) = E[E(X|Y)]\].

In our case, \(E[E(X_2|X_1)]\) represents the expectation of \(X_2\)'s expected values, which are conditioned on different values of \(X_1\), weighted by the marginal probabilities of \(X_1\). Put simply, it's an expectation of expectations. This technique is often used to simplify the computation of expected values, especially when dealing with complex probability distributions, and it's a property used to confirm if computations are done correctly, as both methods should produce the same result.

Marginal PMF

The marginal probability mass function (pmf) is the probability distribution of a subset of a collection of random variables. It gives the probabilities of various values of one random variable irrespective of the values taken by other random variables.

For our joint pmf \(p(x_{1}, x_{2})\), the marginal pmf of \(X_1\), denoted \(p(x_1)\), is calculated by summing the joint pmf over all possible values of \(X_2\):
\[p(x_1) = \sum_{x_2} p(x_1, x_2)\].

This formula provides us with the probabilities only in terms of \(X_1\), which can then be used to compute expected values or conditional probabilities involving \(X_1\) alone. It’s these foundational aspects of probability theory that enable us to extract meaningful information from complex data sets.