Question

Show that the expectation of the sum of two random variables defined over the same sample space is the sum of the expectations. Hint: Let \(p_{1}, p_{2}, p_{3}, \cdots, P_{*}\) be the probabilitics associated with the \(n\) sample points; let \(x_{1}, x_{2}, \cdots, x_{n}\), and \(y_{1}, y_{2}, \cdots, y_{n}\), be the values of the random variables \(x\) and \(y\) for the \(n\) sample points. Write out \(E(x), E(y)\), and \(E(x+y)\).

Short answer

The expectation of the sum of two random variables is the sum of their expectations: \( E(x + y) = E(x) + E(y) \).

Step-by-step solution

Define the Expectation of a Random Variable

The expectation (or expected value) of a random variable is the weighted average of all possible values that the random variable can take, with the probabilities as weights. For a random variable \(x\) with values \(x_1, x_2, \, ... , x_n\) and corresponding probabilities \(p_1, p_2, \, ... , p_n\), the expectation is given by: \[ E(x) = \sum_{i=1}^{n} p_i x_i \]

Write Out the Expectation of \(x\)

Using the definition from Step 1, the expectation of the random variable \(x\) is: \[ E(x) = \sum_{i=1}^{n} p_i x_i \]

Write Out the Expectation of \(y\)

Similarly, for another random variable \(y\) with values \(y_1, y_2, \, ... , y_n\) and corresponding probabilities \(p_1, p_2, \, ... , p_n\), the expectation is given by: \[ E(y) = \sum_{i=1}^{n} p_i y_i \]

Define the Sum of the Two Random Variables

The sum of the two random variables \(x\) and \(y\) at each sample point can be written as \(x_i + y_i\). We need to find the expectation of the sum \( E(x + y) \).

Write Out the Expectation of \(x + y\)

Using the linearity of expectation, we have: \[ E(x + y) = \sum_{i=1}^{n} p_i (x_i + y_i) \]

Apply the Distributive Property

Apply the distributive property to the sum in the expression for \( E(x + y) \): \[ E(x + y) = \sum_{i=1}^{n} p_i x_i + \sum_{i=1}^{n} p_i y_i \]

Use Definition of Expectation

Recognize that the two sums can be rewritten as the expectations of \(x\) and \(y\) respectively: \[ E(x + y) = E(x) + E(y) \]

Conclusion: State the Result

We have shown that the expectation of the sum of two random variables is equal to the sum of the expectations of those two random variables, i.e., \[ E(x + y) = E(x) + E(y) \]

Key concepts

Probability Theory

Probability theory is a branch of mathematics that deals with the analysis of random phenomena. The outcome of a random event can be one of several possible results, and the likelihood of each result is quantified by a probability. Probability values range from 0 (indicating an impossible event) to 1 (indicating a certain event). For instance, flipping a fair coin has two possible outcomes, heads or tails, each with a probability of 0.5.
A key part of probability theory involves understanding how different probabilities relate and interact, such as through the concepts of independent and dependent events. It's foundational for comprehending random variables and their behaviors, which leads us into the next important concept.

Linearity of Expectation

Linearity of expectation is a fundamental and very useful principle in probability theory. It states that for any two random variables, the expected value (also known as the expectation) of their sum is equal to the sum of their individual expected values. Mathematically, this can be expressed as: Note that this property does not require the random variables to be independent. This normalization makes expectation calculations much easier and is crucial in fields like statistics, economics, and data science.

Random Variables

A random variable is a numerical description of the outcome of a statistical experiment. There are two main types of random variables: discrete and continuous.

• Discrete random variables can only take on a finite or countably infinite number of distinct values. Examples include the number of heads in a series of coin flips, or the outcome of rolling a six-sided die.
• Continuous random variables can take an infinite number of possible values within a given range. Examples are the height of students in a class, or the time it takes for a computer algorithm to run.

For both types, the expectation (or expected value) is a weighted average of the possible values, with the weights being the probabilities of those values. Mathematically, for a discrete random variable, the expectation is given by:

For a continuous random variable, it is given by: Understanding these concepts is essential for analyzing how random variables behave and for calculating important metrics such as variances, covariances, and standard deviations.