Question

Recall that two events $A$ and $B$ are called independent if $p(AB)=p(A)p(B).$ Similarly two random variables $x$ and $y$ are called independent if the joint probability function $f(x,y)=g(x)h(y).$ Show that if $x$ and $y$ are independent, then the expectation or average of $xy$ is $E(xy)=E(x)E(y)={\mu }_x{\mu }_y$.

Short answer

If X and Y are independent, then $E(XY)=E(X)E(Y)={\mu }_x{\mu }_y$.

Step-by-step solution

Define Independence of Random Variables

Given two random variables, X and Y, they are independent if their joint probability function is the product of their individual probability functions: $f(x,y)=g(x)h(y)$.

Define Expectation of a Function of Random Variables

The expectation of the product of X and Y is defined as $E(XY)$. By definition, this means integrating the product $x\ast y$ over the joint probability density function $f(x,y)$:$$E(XY)=\int \int xyf(x,y)dxdy$$

Substitute Independence Condition

Given that X and Y are independent, substitute $f(x,y)=g(x)h(y)$ into the expectation formula:$$E(XY)=\int \int xyg(x)h(y)dxdy$$

Separate the Integrals

Separate the double integral into the product of two single integrals: $$E(XY)=\int xg(x)dx\int yh(y)dy$$

Recognize Each Integral as an Expectation

Recognize that $\int xg(x)dx$ is the definition of the expected value of X, $E(X)={\mu }_x$, and $\int yh(y)dy$ is the expected value of Y, $E(Y)={\mu }_y$. Thus,$$E(XY)=E(X)E(Y)={\mu }_x{\mu }_y$$

Key concepts

Joint Probability Function

The concept of a joint probability function is essential when dealing with multiple random variables. Simply put, it's a function that gives us the probability that each of the variables falls within a specific range or takes on certain values simultaneously. For two random variables, X and Y, their joint probability function is denoted as $f(x,y)$. If X and Y are independent, the joint probability function can be expressed as the product of their individual probability functions: $f(x,y)=g(x)h(y)$. This means that the probability of X and Y occurring together is just the product of their separate probabilities. The joint probability function is crucial for understanding interactions between variables, particularly when determining their combined behavior.

Expectation of a Function

The expectation of a function of random variables provides us with the long-term average or mean value you would expect if an experiment were repeated many times. For example, the expectation of the product of X and Y, denoted as $E(XY)$, is simply the average value of the product over all possible values of X and Y. Mathematically, this is expressed as \int \int xy f(x, y) \, dx \, dy\
.
This integral sums up the product $xy$ weighted by their joint probabilities over the possible values of X and Y. For independent variables, substituting $f(x,y)=g(x)h(y)$ allows breaking down the process into simpler, separate calculations for each variable.

Integration of Probability Density

The integration of a probability density function helps us find the expectation of a random variable or any function involving random variables. If you want to calculate the expectation of the product of independent random variables X and Y, you'd integrate their product over the joint density function:

$E(XY)=\int \int xyf(x,y)dxdy$.

Substituting $f(x,y)=g(x)h(y)$ since X and Y are independent, we get:

$E(XY)=\int \int xyg(x)h(y)dxdy$.

The integration separates into:

$E(XY)=(\int xg(x)dx)(\int yh(y)dy)$.

Essentially, this breaks the problem into finding the expectations of X and Y separately and then multiplying these results. Integration in this context simplifies computing expectations for independent variables.

Expected Value

The expected value or expectation of a random variable is a fundamental concept in probability and statistics. It gives you a measure of the 'central' tendency or the mean value you'd anticipate over many trials. For any random variable X, its expected value, $E(X)$, is computed as: $E(X)=\int xg(x)dx$.

Similarly, for Y, the expected value is: $E(Y)=\int yh(y)dy$.

If X and Y are independent random variables, the expected value of their product is simply the product of their individual expected values:

$E(XY)=E(X)E(Y)={\mu }_x{\mu }_y$.

This result is very useful when dealing with complex systems as it allows for simplifying calculations assuming independence of the involved variables. Understanding expected values is key to making predictions and informed decisions based on probabilistic models.