Question

Let \(S^{2}\) be the sample variance of a random sample of size \(n>1\) from \(N(\mu, \theta), 0<\theta<\infty\), where \(\mu\) is known. We know \(E\left(S^{2}\right)=\theta\). (a) What is the efficiency of \(S^{2} ?\) (b) Under these conditions, what is the mle \(\widehat{\theta}\) of \(\theta ?\) (c) What is the asymptotic distribution of \(\sqrt{n}(\widehat{\theta}-\theta) ?\)

Short answer

The efficiency of \(S^{2}\) is 1. The Maximum Likelihood Estimation (MLE) for \(\theta\) is \(S^{2}\). The asymptotic distribution of \(\sqrt{n}(\widehat{\theta}-\theta)\) is normal distribution with mean 0 and variance \(\frac{1}{\theta}\).

Step-by-step solution

Calculate Efficiency

Efficiency of an estimator is typically shown by comparing the variances of two unbiased estimators. Here, since we are talking about sample variance (which is an unbiased estimator of population variance), its efficiency is 1 because there is no other estimator to compare with.

Find MLE of theta

In this case, it's given that the sample is drawn from a normal distribution with known mean \(\mu\). The likelihood function \(\mathcal{L}(\mu,\theta|X) = \frac{1}{\sqrt{2\pi\theta}^{n}} e^{\frac{-1}{2\theta}\sum_{i=1}^{n}(x_i-\mu)^2}\). Note that we treat \(X\) and \(\mu\) as given and \(\theta\) as the variable we want to optimize. Take the derivative \(\frac{\partial}{\partial\theta}\), set it to zero and solve for \(\theta\). We find the Maximum Likelihood Estimation (MLE) \(\widehat{\theta} = S^{2}\).

Find Asymptotic Distribution

The asymptotic distribution of the MLE can be found by calculating the limiting distribution as \(n\) approaches infinity. For large \(n\), by the central limit theorem, \(\sqrt{n}(\widehat{\theta}-\theta)\) will be normally distributed with mean 0 and variance equal to the Fisher information \(\frac{1}{\theta}\).

Key concepts

Efficiency of Estimators

When evaluating the performance of statistical estimators, the concept of efficiency emerges as a critical measure. At its heart, efficiency is a comparison of the variance among unbiased estimators. An efficient estimator is one which has the smallest variance. This feature is particularly important because it directly affects the precision of the estimates produced. In simple terms, among all unbiased estimators for a given parameter, the most efficient one squanders the least amount of information about the parameter.

In the context of our exercise, the sample variance, denoted as \( S^2 \), is an unbiased estimator of the population variance \( \theta \) for a normal distribution. Unbiasedness is a desirable property where the mean of the estimator's sampling distribution equals the true parameter value. Here, the efficiency of \( S^2 \) is suggested to be one (the maximum), meaning it's as efficient as it gets because there isn't an alternative unbiased estimator with a smaller variance for comparison. Essentially, it implies that \( S^2 \) makes full use of the sample information to estimate the population variance.

Maximum Likelihood Estimation (MLE)

Maximum Likelihood Estimation (MLE) represents a fundamental technique in the realm of statistics for estimating the parameters of a model. The premise of MLE is to find the parameter values that make the observed data most probable. This approach revolves around the likelihood function, which gauges the probability of the observed data under different parameter values.

In our discussed problem, the likelihood function is derived from the normal distribution with a known mean \( \text{\( \boldsymbol{\mu} \)} \). By taking the natural logarithm of the likelihood, differentiating with respect to \( \theta \), and setting this derivative equal to zero, we determine the value that maximizes this function. The MLE for the variance \( \theta \) in this scenario turns out to be the sample variance \( S^2 \), uncovered through this calculus-based optimization process. This result reflects the MLE’s ability to adapt to the specific characteristics of the data it's derived from, reinforcing its standing as a versatile and powerful method for estimation in statistical analysis.

Asymptotic Distribution

Asymptotic distribution is a concept that often surfaces in the study of statistics. It encapsulates the behavior of an estimator or a statistic as the sample size becomes very large. More formally, the term refers to the distribution toward which the estimator's probability distribution converges as the sample size approaches infinity.

Turning to the context of our example regarding the asymptotic distribution of the MLE, \( \text{\( \boldsymbol{\widehat{\theta}} \)} \), we draw on the central limit theorem to support the assertion that \( \text{\( \boldsymbol{\sqrt{n}(\boldsymbol{\widehat{\theta}}-\theta)} \)} \) will follow a normal distribution as \( n \), the sample size, grows indefinitely. The mean of this distribution will be zero, reflecting the fact that the MLE is unbiased. The variance, by extension, will correspond to the inverse of the Fisher information, which is \( 1/\theta \) for a normal distribution. This distribution helps us approximate the distribution of the estimator for large sample sizes, a key advantage when conducting statistical inference for complex models.