Question

Show that the mean \(\bar{X}\) of a random sample of size \(n\) from a distribution having pdf \(f(x ; \theta)=(1 / \theta) e^{-(x / \theta)}, 0

Short answer

The mean \(\bar{X}\) of a random sample of size \(n\) from the given distribution is indeed an unbiased estimator of \(\theta\) and it has a variance given by \(\theta^{2} / n\).

Step-by-step solution

Calculate the expectation of X

The expectation of \(X\), \(E[X]\), is given by \(\int_{-\infty}^{\infty} x f(x ; \theta) dx\), where \(f(x ; \theta)\) is the PDF. Plugging in the given PDF, the expectation can be computed by solving \(\int_{0}^{\infty} x \frac{1}{\theta} e^{-(x / \theta)} dx\). This integral equals \(\theta\), which shows that \(E[X] = \theta\).

Proving the Unbiasedness of the Estimator

Now, because \(X1, X2,...,Xn\) are random sample, we have \(\bar{X} = \frac{1}{n} \sum_{i=1}^{n} Xi\) and thus \(E[\bar{X}] = \frac{1}{n} \sum_{i=1}^{n} E[Xi]\) by linearity of expectation. Each \(E[Xi]\) is \(\theta\), by what we found in step 1. This means that \(E[\bar{X}] = \theta\), proving that \(\bar{X}\) is an unbiased estimator of \(\theta\).

Calculate the Variance

The variance of \(X\), \(Var[X]\), is given by \(E[X^2] - (E[X])^2\). We know that \(E[X] = \theta\), and \(E[X^2]\) is found to be \(2 \theta^2\) by computing the integral \(\int_{0}^{\infty} x^2 \frac{1}{\theta} e^{-(x / \theta)} dx\). Then we compute \(Var[X] = 2 \theta^2 - \theta^2 = \theta^2\). For \(Var[\bar{X}]\), we know that the variance of the mean of \(n\) i.i.d random variables is \(Var[\bar{X}] = \frac{1}{n^2} \sum_{i=1}^{n} Var[Xi]\) and each \(Var[Xi] = \theta^2\). Hence, \(Var[\bar{X}] = \frac{\theta^2}{n}\). This completes the proof.

Key concepts

Expectation of a Random Variable

In probability theory, the expectation of a random variable, often denoted by \( E[X] \), is a fundamental concept that represents the average value you would expect to see if you could repeat an experiment over and over again, indefinitely. It is the probability-weighted average of all possible values. Think of it like the center of gravity for the distribution of outcomes.

For a continuous random variable with a probability density function (pdf) \( f(x; \theta) \), we find the expectation by integrating over all possible values:
\[ E[X] = \int_{-\infty}^{\infty} x f(x; \theta) dx \],
In the given exercise, the random variable \(X\) has a specific pdf which is defined only for positive values of \(x\). Thus, the limits of the integral are from 0 to \(\infty\), corresponding to the support of \(X\). By integrating \(x\) multiplied by the pdf, we essentially calculate a weighted average, where each value of \(x\) is weighted by its probability. This process gives us a single value, \(\theta\), revealing that our estimator \(\bar{X}\) correctly averages to the parameter we aim to estimate.

Variance of an Estimator

Variance measures how much the values of a random variable spread out from their mean. The concept of variance is crucial in statistics as it provides insights into the reliability and precision of an estimator. In the context of an estimator, which is a function of random variables created to estimate some parameter, its variance gauges how much the estimates will vary from one sample to another.

The variance of an estimator \(\bar{X}\) is symbolized by \(Var[\bar{X}]\) and it quantifies the dispersion of the estimator’s distribution. To compute it, we use the following relationship:
\[ Var[\bar{X}] = E[\bar{X}^2] - (E[\bar{X}])^2 \].
In the case of independent and identically distributed (i.i.d.) variables, the variance of the mean of these variables is \(Var[\bar{X}] = \frac{1}{n^2} \sum_{i=1}^{n} Var[Xi]\), which simplifies to \(\frac{Var[X]}{n}\) when each \(Xi\) has the same variance. When an estimator has low variance, it indicates that the estimated values tend to be close to each other and to the true parameter value. In the exercise, we learned that \(Var[\bar{X}] = \frac{\theta^2}{n}\), marking the mean \(\bar{X}\) as a more precise estimator as the sample size \(n\) increases.

Integral Calculus in Statistics

Integral calculus provides powerful tools for statisticians, particularly when dealing with the continuous probability distributions. It allows us to calculate various properties of random variables that are described by a pdf, such as their expectation, variance, and probabilities of certain outcomes.

The use of integration in the context of the given exercise is a perfect illustration. By integrating the function \(x \times f(x; \theta)\) across its domain, we effectively calculate the expected value of the random variable \(X\), and by integrating \(x^2 \times f(x; \theta)\), we can find the second moment, which is utilized in determining the variance.

Integral calculus becomes indispensable when we need to summarize an entire range of values with continuous probabilities. Through the integration of the pdf over a particular interval, we can even determine the probability that a random variable will fall within that range. The simplicity of the exponential distribution in this example illustrates how the integration process allows us to derive key statistical properties that define the foundations of theoretical statistics and inform practical data analysis.