Question

Let \(X_{1}, X_{2}, \ldots, X_{n}\) denote a random sample from a normal distribution with mean zero and variance \(\theta, 0<\theta<\infty\). Show that \(\sum_{1}^{n} X_{i}^{2} / n\) is an unbiased estimator of \(\theta\) and has variance \(2 \theta^{2} / n\)

Short answer

The estimator \( \sum_{1}^{n} X_{i}^{2} / n \) is an unbiased estimator of \( \theta \) and its variance is \( 2\theta^{2}/n \).

Step-by-step solution

Define the estimator

We analyze and define the estimator given, which is \( \sum_{1}^{n} X_{i}^{2} / n \)

Prove that the estimator is unbiased

To prove that an estimator \( \hat{\theta} \) is unbiased for \( \theta \), we need to show that \( E[\hat{\theta}] = \theta \). If \( X_{i} \) are normally distributed with mean 0 and variance \( \theta \), then \( E[X_{i}^{2}] = \theta \), because the second moment of a normal distribution is the variance. So the expected value of the estimator \( E[\hat{\theta}] \) is \( E[\sum_{1}^{n} X_{i}^{2} / n] = \sum_{1}^{n} E[X_{i}^{2}] / n = n\theta / n = \theta \). So, our estimator is unbiased.

Calculate the variance of the estimator

Next, we need to calculate the variance \( Var(\hat{\theta}) \). Variance of sum of square of standard normal random variable \( X_{i}^{2} \) is \( 2 \). Thus, the variance of \( X_{i}^{2} \) is \( 2\theta^{2} \). Using the property that variance of sum of random variables is the sum of their variances, we get \( Var(\sum_{1}^{n} X_{i}^{2}) = n*2\theta^{2} \). Hence, \( Var(\hat{\theta}) = Var(\sum_{1}^{n} X_{i}^{2} / n) = 2\theta^{2}/n \).

Key concepts

Normal Distribution

A normal distribution, also known as a Gaussian distribution, is one of the most crucial probability distributions in statistics. It is defined by its bell-shaped curve, which is symmetric around the mean. In mathematical terms, a random variable following a normal distribution can be expressed as:

• Mean (\( ext{mean} = heta \) for our problem)
• Variance (\( ext{variance} = heta \) since the variance is tied to \( heta \) in this context)

The mean indicates the center or peak of the distribution, while the variance measures the spread or dispersion of data points around the mean.
In our exercise, each sample \( X_i \) is drawn from a normal distribution with a specified mean (\( 0 \)) and variance (\( heta \)). This implies that each data set of \( X_i^2 \) contributes to both estimating the mean \( 0 \) and the unknown variance \( heta \). The property of the normal distribution that supplements this analysis is how these sample moments relate to the theoretical moments.

Estimator Bias

Understanding estimator bias involves checking if an estimator consistently targets the parameter of interest, in this case, \( heta \). An unbiased estimator has an expected value equal to the population parameter it estimates. For our exercise, we want to see if \( \hat{\theta} = \sum_{1}^{n} X_{i}^{2} / n \) is unbiased for \( \theta \).
Steps to determine unbiased nature:

• Calculate the expected value of the estimator: {% raw %}\( E[\hat{\theta}] = E\left[\sum_{1}^{n} X_{i}^{2} / n\right] = \sum_{1}^{n} E[X_{i}^{2}] / n = \theta \), since each \( X_i \) has mean 0 and variance \( \theta \).
• This confirms that \( \hat{\theta} \) is unbiased because \( E[\hat{\theta}] = \theta \).

An unbiased estimator is desirable in statistics because it signals accuracy in aiming for the true parameter, avoiding systematic overestimation or underestimation.

Variance of Estimator

The variance of an estimator is crucial for understanding its precision. A low variance indicates that the estimator produces similar outcomes across different samples, leading to accurate predictions. For the estimator \( \hat{\theta} = \sum_{1}^{n} X_{i}^{2} / n \), we identify its variance as follows:

• First, understand that \( Var(X_i^2) = 2\theta^2 \).
• The variance of a sum of independent, identically distributed random variables is the sum of their variances, leading to \( Var\left(\sum_{1}^{n} X_{i}^{2}\right) = n(2\theta^2) \).
• Hence, \( Var(\hat{\theta}) = Var\left(\sum_{1}^{n} X_{i}^{2} / n\right) = 2\theta^2 / n \).

This indicates that the precision of the estimator improves with an increasing sample size \( n \). The decrease in variance with larger \( n \) follows the principle that larger sample sizes provide more reliable estimates. This insight shows that our estimator \( \hat{\theta} \) is not only unbiased but also efficient as sample size increases.