Question

Let \(X, X_{1}, X_{2}, \ldots\) be random variables having finite second moments. Show that \(\lim _{n \rightarrow \infty}\left\|X_{n}-X\right\|=0\), if and only if both conditions \(\lim _{n \rightarrow \infty} E\left[X_{n} Y\right]\) \(=E[X Y]\) for all random variables \(Y\) satisfying \(E\left[Y^{2}\right]<\infty\), and \(\lim _{n \rightarrow \infty}\) \(\left\|X_{n}\right\|=\|X\|\) hold.

Short answer

To show the equivalence between the given conditions and the convergence of the sequence of random variables \(X_n\) to the random variable \(X\) in mean square sense, we first prove that if \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\), then \(\lim_{n \rightarrow \infty} E[X_n Y] = E[XY]\) for all random variables \(Y\) satisfying \(E[Y^2] < \infty\) and \(\lim_{n \rightarrow \infty} \|X_n\| = \|X\|\). Next, we prove the converse, if \(\lim_{n \rightarrow \infty} E[X_n Y] = E[XY]\) for all \(Y\) satisfying \(E[Y^2] < \infty\) and \(\lim_{n \rightarrow \infty} \|X_n\| = \|X\|\), then \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\).

Step-by-step solution

Part 1: Convergence of \(X_n\) to \(X\) implies the conditions hold

Suppose that \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\). We need to show that the conditions stated in the problem hold. (1) Let \(Y\) be a random variable with \(E[Y^2] < \infty\). Then, using Cauchy-Schwarz inequality, \(|E[(X_n - X)Y]|^2 \le E[(X_n - X)^2]E[Y^2]\). Since \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\), we have \(\lim_{n\rightarrow\infty}E[(X_n - X)^2] = 0\), and therefore \(\lim_{n\rightarrow\infty}|E[(X_n - X)Y]|^2 = 0\). It follows that \(\lim_{n\rightarrow\infty}|E[X_nY] - E[XY]| = 0\) and thus \(\lim_{n\rightarrow\infty}E[X_nY] = E[XY]\). (2) Since \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\), we have \[ \lim_{n\rightarrow\infty}\|X_n\|^2 = \lim_{n\rightarrow\infty}\|X\|^2 + 2E[(X_n - X)X] - E[(X_n - X)^2]. \] Recall that we have shown in (1) that \(\lim_{n\rightarrow\infty}E[X_n Y] = E[XY]\) for all \(Y\) satisfying \(E[Y^2] < \infty\). So, in particular, we can choose \(Y = X\) and have \(\lim_{n\rightarrow\infty}E[X_n X] = E[X^2]\). As a consequence, \(\lim_{n\rightarrow\infty}\|X_n\|^2 = \|X\|^2\) and thus \(\lim_{n\rightarrow\infty}\|X_n\| = \|X\|\).

Part 2: The conditions imply convergence of \(X_n\) to \(X\)

Suppose that \(\lim_{n\rightarrow\infty}E[X_nY] = E[XY]\) for all random variables \(Y\) satisfying \(E[Y^2] < \infty\), and also that \(\lim_{n\rightarrow\infty}\|X_n\| = \|X\|\). We want to show that \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\). To do this, let's consider \[ \|X_n - X\|^2 = E[(X_n - X)^2] = \|X_n\|^2 + \|X\|^2 - 2E[X_nX]. \] Since \(\lim_{n\rightarrow\infty}\|X_n\| = \|X\|\), we have \(\lim_{n\rightarrow\infty}\|X_n\|^2 = \|X\|^2\). Additionally, as shown in the problem statement, if we choose \(Y = X\), the condition implies that \(\lim_{n\rightarrow\infty}E[X_nX] = E[X^2]\). Therefore, we have \[ \lim_{n\rightarrow\infty}\|X_n - X\|^2 = \|X\|^2 + \|X\|^2 - 2E[X^2] = 0, \] and thus, \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\). With the successful completion of both parts, we have shown that the given conditions are equivalent to the convergence of the sequence \(X_n\) to the random variable \(X\) in mean square sense (i.e. \(\lim_{n\rightarrow\infty}\|X_n - X\| = 0\)).

Key concepts

Random Variables

In the world of probability and statistics, a random variable is a fundamental concept that represents a numerical quantity determined by the outcome of a random event. Different outcomes may occur with certain probabilities, and these outcomes dictate the value that the random variable takes on.

Imagine you're rolling a dice; the result could be any number between 1 to 6. Each of these results is an example of a random variable. In fact, there are two types of random variables — discrete and continuous. The roll of a dice is a discrete random variable because it can take on one of a countable number of values. Continuous random variables, however, can represent any value within an interval due to measurements or calculations — think of measuring the time it takes for milk to spoil at a certain temperature.

Understanding random variables is crucial for gauging mean square convergence, as it involves the behavior of sequences of random variables as they approach a limit in a particular probabilistic sense.

Second Moments

Diving deeper into the characteristics of random variables, the second moment plays a significant role. It's essentially a measurement of a random variable's variation around its mean, denoted as the expected value of the squared deviation from the mean: \( E[(X - E[X])^2] \).

This concept is widely known as the variance, and it provides valuable insight into the 'spread' of a set of random variables. The square of the standard deviation — which indicates how 'spread out' the values of a random variable are — is an example of a second moment.

In the context of our exercise, the second moments must be finite, implying that the random variables do not have extreme values with high probabilities. This finiteness is critical for the use of certain inequalities and limits, including proving mean square convergence of a sequence of random variables.

Cauchy-Schwarz Inequality

The Cauchy-Schwarz inequality is a powerful and elegant piece of mathematics within the field of linear algebra. It describes a relationship between the dot product (or the expected value in probability) of two vectors and the product of their magnitudes.

The inequality states that for any two vectors in an inner product space, the absolute value of their inner product is less than or equal to the product of their norms: \( |\langle u, v\rangle| \leq \|u\| \cdot \|v\| \).

Applied to random variables, it becomes: \( |E[XY]|^2 \leq E[X^2]E[Y^2] \), assuming that the expected values exist. This can be seen as a kind of 'guardrail' ensuring that the expected value of the product doesn't become unmanageably large compared to the respective second moments (variances) of the individual random variables.

The interpretation of Cauchy-Schwarz in our exercise is critical. It's used in proving that the mean square convergence of a sequence of random variables implies certain conditions regarding the expected products with other random variables. This inequality ensures that, under the square mean convergence criteria, the correlation between sequences of random variables is bounded and can be related to their individual variances, which is a concept pivotal in advanced statistics and probability theory.