Question

Let \(S^{2}\) be the sample variance of a random sample of size \(n>1\) from \(N(\mu, \theta), 0<\theta<\infty\), where \(\mu\) is known. We know \(E\left(S^{2}\right)=\theta\) (a) What is the efficiency of \(S^{2} ?\) (b) Under these conditions, what is the mle \(\widehat{\theta}\) of \(\theta\) ? (c) What is the asymptotic distribution of \(\sqrt{n}(\widehat{\theta}-\theta) ?\)

Short answer

The efficiency of \(S^{2}\) in this case will depend on the actual value of the variance of \(S^{2}\), and it is obtained by dividing \(\theta^{2}\) by this variance. The MLE (\(\widehat{\theta}\)) of \(\theta\) can be found by differentiating the log-likelihood function and setting it to zero. Finally, the asymptotic distribution of \(\sqrt{n}(\widehat{\theta}-\theta)\) is a normal distribution with mean zero and variance equal to the variance of the MLE divided by n.

Step-by-step solution

Calculate the Variance and Efficiency of \(S^{2}\)

We start by calculating the variance of the sample variance, Var(\(S^{2}\)). This can be done using the formula Var(\(S^{2}\)) = \(E[(S^{2} - \theta)^{2}]\). Once we have the variance, we find the efficiency of \(S^{2}\) by dividing \(\theta^{2}\) by this variance. This gives us the efficiency of \(S^{2}\) as an estimator of \(\theta\).

Find the MLE of \(\theta\)

The likelihood function for a sample from a Normal distribution with known mean and unknown variance is \(\prod_{i=1}^{n} 1/\sqrt{2 \pi \theta} exp[ -\frac{1}{2\theta} (x_{i}-\mu)^{2}]\). Next, we take the natural log to get the log-likelihood function, differentiate this with respect to \(\theta\) and set the derivative equal to zero to solve for \(\theta\). The resulting value of \(\theta\) is our MLE, \(\widehat{\theta}\).

Find the Asymptotic Distribution of \(\sqrt{n}(\widehat{\theta}-\theta)\)

By the Central Limit Theorem, the distribution of \(\sqrt{n}(\widehat{\theta}-\theta)\) tends towards a normal distribution as n gets large. To find the actual distribution, we need to calculate its mean and variance. The mean is zero since the mean of \(\widehat{\theta}\) is \(\theta\). The variance can be obtained by first calculating the variance of the MLE, and then dividing this by n.

Key concepts

Estimation Efficiency

Understanding estimation efficiency is crucial to evaluating how well a statistical estimator performs. Efficiency is a measure of the quality of an estimator in terms of the variance of the estimated parameters. In the context of the sample variance, denoted as \(S^{2}\), its efficiency directly relates to how closely it estimates the population variance \(\theta\).

Given a sample from a normal distribution \(N(\mu, \theta)\), where \(\mu\) is known, the expected value of the sample variance, \(E(S^{2})\), is equal to the population variance \(\theta\). To calculate the efficiency of \(S^{2}\), you need to assess the variance of \(S^{2}\). The efficiency of an estimator can then be quantified by comparing its variance to the variance of an 'ideal' estimator with the lowest possible variance, known as the Cramér-Rao lower bound.

The formula for efficiency is the ratio of the variance of the 'ideal' estimator to the actual variance of the estimator being analyzed. Hence, if the sample variance is an efficient estimator, its variance will be close to the Cramér-Rao lower bound, resulting in an efficiency value near one. A higher efficiency signifies a more 'reliable' estimator because it implies less variability and therefore more precise estimation of the population parameter.

Maximum Likelihood Estimation (MLE)

Maximum Likelihood Estimation (MLE) is a method used in statistics to estimate the parameters of a statistical model. It operates under the principle of choosing the parameter values that maximize the likelihood function, which measures the probability of the observed data given certain parameter values.

In the given exercise about sample variance, the MLE for the population variance \(\theta\) is computed under the assumption that the mean \(\mu\) is known. The likelihood function for this scenario involves the product of the probabilities of each observation, assuming they are independently drawn from a normal distribution. To find the MLE, \(\widehat{\theta}\), the natural logarithm of the likelihood function is used to transform the multiplicative problem into an additive one, simplifying the differentiation required to solve for \(\theta\).

This approach highlights the adaptive nature of MLE; it molds to the structure of the data and distribution involved, which, when applied correctly, yields an estimator that is consistent (converges to the true parameter value as sample size increases) and efficient (achieves the smallest possible variance among unbiased estimators) for large samples.

Asymptotic Distribution

The concept of an asymptotic distribution comes into play when examining the behavior of a statistical estimator as the sample size grows indefinitely. It provides an approximation of the distribution of the estimator when the number of observations is large.

In regard to the exercise, we are interested in the asymptotic distribution of \(\sqrt{n}(\widehat{\theta}-\theta)\), where \(\widehat{\theta}\) is the MLE of the population variance. The Central Limit Theorem (CLT) underpins this concept and assures that for a large sample size \(n\), \(\sqrt{n}(\widehat{\theta}-\theta)\) will follow a normal distribution with mean 0 and a certain variance that is derived from the variance of the MLE. The use of the CLT is especially helpful because it allows us to make probabilistic statements about how close \(\widehat{\theta}\) is to \(\theta\) for large values of \(n\).

This distribution is termed 'asymptotic' because it describes the limiting behavior of the estimator's distribution, which forms the foundation for constructing confidence intervals and hypothesis tests for the population parameter based on the MLE.