Question

Show that for random variables $X$ and $Z$.

$E\left[(X-Y{)}^2\right]=E\left[X^2\right]-E\left[Y^2\right]$

where$Y=E[X\mid Z]$

Short answer

$X-E(X\mid Z)$ is orthogonal to $E(X\mid Z)$.

Step-by-step solution

Given Information

$Y=E[X\mid Z]$

Explanation

Write out the left side. We have that

$E\left((X-Y{)}^2\right)=E\left(X^2+Y^2-2XY\right)$

$=E\left(X^2\right)+E\left(Y^2\right)-2E\left(XY\right)$

Now, what is $E(XY)$? Let's prove that it is equal to $E\left(Y^2\right)$.

$E\left(XY-Y^2\right)=E\left(Y\right(X-Y\left)\right)$

$=E\left(E\right(X\mid Z)·(X-E(X\mid Z)\left)\right)=0$

Explanation

The expression above is equal to zero, since we know that the projection $E(X\mid Z)$and the connection between the point $X$and the projection $E(X\mid Z)$are orthogonal. Therefore $E\left(XY\right)=E\left(Y^2\right)$which implies

$E\left((X-Y{)}^2\right)=E\left(X^2\right)-E\left(Y^2\right)$.

Final Answer

$X-E(X\mid Z)$ is orthogonal to $E(X\mid Z)$.