Question

Prove that \(E\left[X^{2}\right] \geq(E[X])^{2}\). When do we have equality?

Short answer

Using Jensen's Inequality on the convex function \(f(x) = x^2\), we obtain the inequality \(E[X^2] \geq (E[X])^2\). The equality holds if and only if X is a constant random variable.

Step-by-step solution

Identify the convex function

We will use the convex function f(x) = x^2. A function is convex if it satisfies the following condition for any two points x, y and any t ∈ (0, 1): f(tx + (1-t)y) ≤ tf(x) + (1-t)f(y) In our case, f(x) is a quadratic function with a positive leading coefficient, which makes it a convex function.

Apply Jensen's Inequality

Now, let's apply Jensen's Inequality to the function f(x) and the random variable X. Since f(x) is convex, we have: \(E[f(X)] \geq f(E[X])\) Substituting f(x) = x^2, we get: \(E[X^2] \geq (E[X])^2\)

Determine when equality holds

The equality in Jensen's Inequality holds if and only if X is a constant random variable or if the function f is linear on the support of the random variable X. In our case, the function f(x) = x^2 is not linear. Therefore, the equality holds if and only if X is a constant random variable. So the inequality \(E[X^2] \geq (E[X])^2\) holds for any random variable X, and we have equality if and only if X is a constant random variable.

Key concepts

Expected Value

In the realm of probability and statistics, the concept of the expected value is pivotal. Think of the expected value, denoted as E[X], as the long-term average or mean of a set of numbers if a process is repeated an infinite number of times. For a random variable, which can take on various values each with its own probability, the expected value is calculated by summing the products of each potential value of the variable and its corresponding probability.

For example, in a simple dice game, the expected value of the outcome when rolling a six-sided die is 3.5. This is because (1+2+3+4+5+6)/6 = 3.5. In the given exercise, understanding the expected value helps us analyze the relationship between the square of a random variable and the square of its expected value.

Convex Function

A convex function is a type of mathematical function with a specific shape and set of properties that are important in optimization problems. Visually, if you imagine drawing a straight line segment between any two points on the graph of a convex function, the segment will always lie above or on the graph. A function f is strictly convex if for any two distinct points x and y and any t in the open interval (0,1), the function satisfies the inequality f(tx + (1-t)y) < tf(x) + (1-t)f(y).

In simpler terms, a bowl-shaped graph represents a convex function, such as f(x) = x^2 for x greater than zero. This property of convex functions plays a critical role in Jensen's Inequality, which is used in the textbook exercise to compare the expected value of a random variable squared with the square of its expected value.

Random Variables

Random variables are fundamental to probability theory. They are not variables in the traditional algebraic sense but rather functions that assign numerical values to each outcome in a sample space of a random process. There are two types of random variables: discrete and continuous.

Discrete random variables have a countable number of possible values, like the result of rolling a die or flipping a coin. Continuous random variables, on the other hand, can take on an infinite number of possible values within a given range, such as the exact amount of rainfall in a day.

Understanding random variables is essential as they bridge the gap between raw random events and numerical values, which can then be analyzed using statistical methods. In the exercise related to Jensen's Inequality, we delve into the behavior of a random variable when subjected to a convex function, which, in this case, is the act of squaring the variable.