Question

\begin{aligned} &\text { (a) Show that }\\\ &\operatorname{Cov}(X, Y)=\operatorname{Cov}(X, E[Y \mid X]) \end{aligned} (b) Suppose, that, for constants \(a\) and \(b\), $$ E[Y \mid X]=a+b X $$ Show that $$ b=\operatorname{Cov}(X, Y) / \operatorname{Var}(X) $$

Short answer

In this problem, we showed that: (a) The covariance between X and Y is equal to the covariance between X and the conditional expectation of Y given X: \( \operatorname{Cov}(X, Y) = \operatorname{Cov}(X, E[Y|X]) \). (b) For the conditional expectation of Y given X, defined as a linear function \( E[Y|X] = a + bX \), the coefficient b is equal to the covariance between X and Y divided by the variance of X: \( b = \frac{\operatorname{Cov}(X, Y)}{\operatorname{Var}(X)} \).

Step-by-step solution

Part (a): Show Cov(X, Y) = Cov(X, E[Y|X])

To prove the equality, we will need to use the definition of covariance and conditional expectation. Recall that: \( \operatorname{Cov}(X, Y) = E[(X - E[X])(Y - E[Y])] \) and \( E[Y|X] = \sum_y y \cdot P(Y=y|X) \, \text{ for all possible values of } X \) We can apply these definitions to prove the result: 1. Write the Cov(X, E[Y|X]): \( \operatorname{Cov}(X, E[Y|X]) = E[(X - E[X])(E[Y|X] - E[E[Y|X]])] \) 2. Rewrite E[Y|X] using Law of Iterated Expectation (E[E[Y|X]] = E[Y]): \( \operatorname{Cov}(X, E[Y|X]) = E[(X - E[X])(E[Y|X] - E[Y])] \) 3. Distribute the expression inside expectation function: \( \operatorname{Cov}(X, E[Y|X]) = E[\{X E[Y|X] - E[Y]X\} - \{XE[X] - E[Y]E[X]\}] \) 4. Use the linearity of expectation: \( \operatorname{Cov}(X, E[Y|X]) = E[XE[Y|X] - YE[X] - XE[Y] + E[Y]E[X]] \) 5. With the definition of covariance, notice that Cov(X, Y) = E[XY] - E[X]E[Y], thus rewrite the expression above: \( \operatorname{Cov}(X, E[Y|X]) = E[(X - E[X])(Y - E[Y])] \) 6. Finally, we can see that Cov(X, E[Y|X]) = Cov(X, Y) Now, let's move on to part (b).

Part (b): Show b = Cov(X, Y) / Var(X)

In this part, we are given that: \( E[Y|X] = a + bX \) We need to show that: \[ b = \frac{\operatorname{Cov}(X, Y)}{\operatorname{Var}(X)} \] To show this, we will apply the previous result in part (a). 1. Recall that Var(X) = Cov(X, X) 2. From part (a) we know that Cov(X, Y) = Cov(X, E[Y|X]) 3. Using E[Y|X] = a + bX, the equation becomes: Cov(X, Y) = Cov(X, a + bX) 4. Apply the linear properties of covariance and simplify: \( Cov(X, Y) = Cov(X, a) + b \cdot Cov(X, X) \) 5. Since a is constant, Cov(X, a) = 0, we get: \( Cov(X, Y) = b \cdot Cov(X, X) \) 6. Dividing both sides by Var(X) = Cov(X, X), we obtain: \( b = \frac{\operatorname{Cov}(X, Y)}{\operatorname{Var}(X)} \) Therefore, this concludes the proof for part (b).