Question

(a) Let \(X\) and \(Y\) be two random variables with finite positive variances. Use the fact that \(\operatorname{var}(a X+Y) \geq 0\), with equality if and only if the linear combination \(a X+Y\) is constant with probability one, to show that \(\operatorname{cov}(X, Y)^{2} \leq \operatorname{var}(X) \operatorname{var}(Y)\); this is a version of the Cauchy-Schwarz inequality. Hence show that \(-1 \leq \operatorname{corr}(X, Y) \leq 1\), and say under what conditions equality is attained. (b) Show that if \(X\) and \(Y\) are independent, \(\operatorname{corr}(X, Y)=0\). Show that the converse is false by considering the variables \(X\) and \(Y=X^{2}-1\), where \(X\) has mean zero, variance one, and \(\mathrm{E}\left(X^{3}\right)=0\)

Short answer

\(\operatorname{cov}(X, Y)^{2} \leq \operatorname{var}(X)\operatorname{var}(Y)\) implies \(-1 \leq \operatorname{corr}(X, Y) \leq 1\). \(\operatorname{corr}(X, Y)=0\) does not imply independence.

Step-by-step solution

Understand the Variance Expression

Consider the variance of the linear combination \(aX + Y\). Given that \(\text{var}(aX + Y) \geq 0\), the variance is zero if and only if \(aX + Y\) is constant with probability one.

Expand the Variance

The variance of \(aX + Y\) is given by:\[\text{var}(aX + Y) = \text{var}(aX) + \text{var}(Y) + 2\,\text{cov}(aX, Y)\]This simplifies to \(a^2\,\text{var}(X) + \text{var}(Y) + 2a\,\text{cov}(X, Y)\).

Use the Non-negativity of Variance

We have \(a^2\,\text{var}(X) + \text{var}(Y) + 2a\,\text{cov}(X, Y) \geq 0\). Both sides of this inequality are quadratic in \(a\).

Complete the Square

Rewrite the inequality as a quadratic in terms of \(a\). If we complete the square, it becomes:\[\left(a\sqrt{\text{var}(X)} + \frac{\text{cov}(X,Y)}{\sqrt{\text{var}(X)}}\right)^2 \geq \frac{\text{cov}(X,Y)^2}{\text{var}(X)} - \text{var}(Y)\]Since the left side is a perfect square and is non-negative, this implies:\[\text{cov}(X,Y)^2 \leq \text{var}(X) \cdot \text{var}(Y)\]which is the Cauchy-Schwarz inequality for covariance.

Define Correlation and Condition for Equality

The correlation \(\text{corr}(X, Y)\) is defined as:\[\text{corr}(X, Y) = \frac{\text{cov}(X, Y)}{\sqrt{\text{var}(X) \cdot \text{var}(Y)}}\]Since \(\text{cov}(X,Y)^2 \leq \text{var}(X) \cdot \text{var}(Y)\), it follows that \(-1 \leq \text{corr}(X, Y) \leq 1\). Equality \(\text{corr}(X, Y) = \pm 1\) occurs if \(Y = aX + b\) for constants \(a\) and \(b\).

Show Zero Correlation if Independent

If \(X\) and \(Y\) are independent, \(\text{cov}(X, Y) = 0\). Thus, by definition, \(\text{corr}(X, Y) = 0\).

Show the Converse is False

Consider \(X\) and \(Y = X^2 - 1\). Since \(X\) has mean zero, variance one, and \(\mathrm{E}\left(X^3\right) = 0\),the covariance \(\text{cov}(X,Y) = \mathrm{E}(X(X^2 - 1)) = \mathrm{E}(X^3 - X) = 0\). Hence, \(\text{corr}(X, Y) = 0\) despite \(X\) and \(Y\) not being independent.

Key concepts

Understanding the Cauchy-Schwarz Inequality in Covariance

The Cauchy-Schwarz Inequality is a crucial concept in statistics and probability, especially when dealing with random variables. It's a mathematical expression that helps us understand the relationship between two random variables by providing an upper bound for their covariance.
In simple terms, it tells us that the square of the covariance between random variables, say \(X\) and \(Y\), is never more than the product of their variances. Formally, this is expressed as:

• \(\operatorname{cov}(X, Y)^2 \leq \operatorname{var}(X) \cdot \operatorname{var}(Y)\)

This inequality holds true for any pair of random variables with finite and positive variances. The equality condition implies that the random variables are perfectly linearly dependent—meaning one can be expressed exactly as a linear function of the other.
This brings us to an important aspect of the correlation coefficient, which we derive from this inequality.

Correlation and Its Bounds

Correlation is a statistical measure that indicates the extent to which two random variables fluctuate together. It is calculated as the covariance of two variables divided by the product of their standard deviations. Expressed as:

• \(\operatorname{corr}(X, Y) = \frac{\operatorname{cov}(X, Y)}{\sqrt{\operatorname{var}(X) \cdot \operatorname{var}(Y)}}\)

Thanks to the Cauchy-Schwarz inequality, we know that correlation values are always between and 1. A correlation of 1 implies perfect positive linear relation, 0 indicates no linear relation, and -1 signifies a perfect negative linear relationship.
The condition for equality, in this case, happens when one variable is a perfect linear transformation of the other, i.e., if \(Y = aX + b\) for some constants \(a\) and \(b\). This means that they follow the same trend very closely, either positively or negatively with respect to the constants.

Independent Random Variables and Zero Correlation

Two random variables are considered independent if the occurrence of any particular value of one variable is not affected by the value of the other. Independence simplifies many statistical calculations, particularly when determining correlations.
If two variables, \(X\) and \(Y\), are independent, this means that their covariance is zero:

• \(\operatorname{cov}(X, Y) = 0\)

Consequently, this also implies that their correlation is zero:

• \(\operatorname{corr}(X, Y) = 0\)

However, a zero correlation does not always translate to independence. There can be instances where random variables have a correlation of zero but are still dependent.
An example given in the exercise explains this using a pair of variables \(X\) and \(Y = X^2 - 1\). Despite having zero correlation, \(X\) and \(Y\) are not independent, highlighting an important distinction between zero correlation and independence.