Question

Let \(X\) be a random variable such that \(E\left[(X-b)^{2}\right]\) exists for all real \(b\). Show that \(E\left[(X-b)^{2}\right]\) is a minimum when \(b=E(X)\).

Short answer

The expression \(E\left[(X - b)^2\right]\) is minimized when \(b\) is equal to the expected value of \(X\) (i.e., when \(b = E[X]\)). This result was found by setting the derivative of the expression with respect to \(b\) equal to zero and solving for \(b\).

Step-by-step solution

Expansion of \(E[(X-b)^{2}]\)

Begin by expanding the term \(E[(X-b)^{2}]\) using the formula \((a - b)^2 = a^2 - 2ab + b^2\). So we have \(E[X^{2} - 2bX + b^{2}]\).

Application of Expected Value Properties

Using the properties of expected values we can break this expression into a sum of expected values: \(E[X^{2}] - E[2bX] + E[b^{2}]\). We can further simplify this by pulling out the constants from each expected value, as is permitted by the properties of expected values, giving us \(E[X^{2}] - 2bE[X] + b^{2}\).

Calculation of Derivative

Next, calculate the derivative of the expression with respect to \(b\) to find the value of \(b\) that minimizes the expression (since the minimum of a function occurs where its derivative is equal to zero). The derivative is \(-2E[X] + 2b\).

Setting derivative equal to zero

Set the derivative equal to zero and solve for \(b\): \(-2E[X] + 2b = 0\). This simplifies to \(b = E[X]\).