Question

Prove that the sample mean is the best linear unbiased estimator of the population mean \(\mu\) as follows. (a) If the real numbers \(a_{1}, a_{2}, \ldots, a_{n}\) satisfy the constraint \(\sum_{i=1}^{n} a_{i}=C\), where \(C\) is a given constant, show that \(\sum_{i=1}^{n} a_{i}^{2}\) is minimised by \(a_{i}=C / n\) for all \(i\). (b) Consider the linear estimator \(\hat{\mu}=\sum_{i=1}^{n} a_{i} x_{i}\). Impose the conditions (i) that it is unbiased, and (ii) that it is as efficient as possible.

Short answer

The sample mean \(\hat{\mu} = \frac{1}{n}\sum_{i=1}^n x_i\) is the best linear unbiased estimator of \(\mu\) because it minimizes \(\sum_{i=1}^n a_i^2\) and satisfies the unbiased condition.

Step-by-step solution

Statement of the Problem

Given real numbers \(a_1, a_2, \ldots, a_n\) must satisfy the constraint \(\sum_{i=1}^n a_i = C\), we need to show that \(\sum_{i=1}^n a_i^2\) is minimized when \(a_i = \frac{C}{n}\) for all \(i\).

Set Up the Minimization Problem

Use the method of Lagrange multipliers to minimize \(\sum_{i=1}^n a_i^2\), subject to the constraint \(\sum_{i=1}^n a_i = C\). The Lagrange function is defined as: \[ L(a_1, a_2, \ldots, a_n, \lambda) = \sum_{i=1}^n a_i^2 + \lambda (C - \sum_{i=1}^n a_i) \]

Compute Partial Derivatives

Calculate the partial derivatives of the Lagrange function with respect to each \(a_i\) and \(\lambda\): \(\frac{\partial L}{\partial a_i} = 2a_i - \lambda = 0 \)\(\frac{\partial L}{\partial \lambda} = C - \sum_{i=1}^n a_i = 0\)

Solve the System of Equations

From \( 2a_i = \lambda \), solve for \(a_i\): \( a_i = \frac{\lambda}{2}\)Substitute into the constraint \(\sum_{i=1}^n a_i = C\) to get \( \sum_{i=1}^n \frac{\lambda}{2} = C \), which simplifies to \( n \cdot \frac{\lambda}{2} = C\)Solve for \(\lambda\): \(\lambda = \frac{2C}{n}\)Thus, \(a_i = \frac{C}{n}\)

Define Linear Estimator

Consider the linear estimator of the population mean, defined as \(\hat{\mu} = \sum_{i=1}^n a_i x_i\). Use the condition that \( \hat{\mu} \) is unbiased and efficient.

Impose Unbiased Condition

For \(\hat{\mu}\) to be unbiased, \( \mathbb{E}[\hat{\mu}] = \mu\). We have \( \mathbb{E}[\hat{\mu}] = \mathbb{E}[\sum_{i=1}^n a_i x_i] = \sum_{i=1}^n a_i \mu = \mu \sum_{i=1}^n a_i \)Since the sum must equal \(1\), \(\sum_{i=1}^n a_i = 1\)

Optimize Efficiency

To achieve maximum efficiency, minimize the variance of \(\hat{\mu}\). Note \( \text{Var}(\hat{\mu}) = \sigma^2 \sum_{i=1}^n a_i^2 \)From part (a), this variance is minimized when \(a_i = \frac{1}{n}\). Thus, \(\hat{\mu} = \sum_{i=1}^n \frac{1}{n} x_i = \frac{1}{n}\sum_{i=1}^n x_i\)

Key concepts

sample mean

The sample mean is the arithmetic average of a set of observations. If you collect data points, such as \( x_1, x_2, \ldots, x_n \), the sample mean is found by summing all data points and then dividing by the number of data points. Mathematically, it is represented as: \[ \bar{x} = \frac{1}{n} \sum_{i=1}^n x_i \]
This calculation provides a single value which represents a central point of the data. It is frequently used in statistics to estimate the population mean. This estimator is crucial because, under certain conditions, it performs exceptionally well with respect to bias and efficiency.

population mean

The population mean, denoted \mu, is the average of all values in a population. Unlike the sample mean, which is computed from a subset of the population data, the population mean encompasses every data point within the entire population. For large data sets or when collecting the whole population's data is feasible (which is often impractical), the population mean can be directly computed:
\[ \mu = \frac{1}{N} \sum_{i=1}^N x_i \]
Here, N represents the total number of observations in the population, and \mu provides a true average value for the population. The population mean is a fixed value, in contrast to the sample mean, which may vary depending on the sample taken.

Lagrange multipliers

Lagrange multipliers are a strategy used in optimization to find local maxima and minima of a function subject to equality constraints. In our specific problem, we wish to minimize the sum of squares of \( a_i \) subject to the constraint \( \sum_{i=1}^n a_i = C \). The Lagrange multiplier technique creates a new function called the Lagrangian, which incorporates this constraint:
\[ L(a_1, a_2, \, \ldots, a_n, \lambda) =\ \sum_{i=1}^n a_i^2 + \lambda(C - \sum_{i=1}^n a_i)
\]

To find minimizing values, we take partial derivatives with respect to each variable (both \( a_i \) and the Lagrange multiplier \( \lambda \)), set them to zero, and solve the resulting equations.

unbiased estimator

An unbiased estimator is a statistical estimator that, on average, gives the true value of the parameter being estimated. In other words, the expected value of an unbiased estimator equals the parameter it estimates. For our linear estimator \( \hat{\mu} = \sum_{i=1}^n a_i x_i \), it is unbiased if
\[ \mathbb{E}[\hat{\mu}] = \mu \]
\(Here, \mathbb{E}[\hat{\mu}] \) is the expected value of \hat{\mu} and should be equal to the population mean \( \mu \). In the context of our estimator, the condition that ensures it is unbiased is
\[ \sum_{i=1}^n a_i = 1 \]

If an estimator consistently hits the true parameter value, it offers more reliable and accurate inferences.

variance minimization

Variance minimization involves finding an estimator that not only is unbiased but also has the smallest possible variance among all unbiased estimators. For our linear estimator \( \hat{\mu} = \sum_{i=1}^n a_i x_i \), the goal is to minimize \( \text{Var}(\hat{\mu}) \), noting that the variance of a sum of independent variables (with equal variance) is given by:
\[ \text{Var}(\hat{\mu}) = \sigma^2 \sum_{i=1}^n a_i^2 \]
Using the result from part (a) of our problem, we know that \(\sum_{i=1}^n a_i^2\) is minimized when each \( a_i \) equals \( \frac{1}{n} \). Hence, this combination yields the lowest variance for \hat{\mu} while ensuring it is still unbiased.
This leads to the conclusion that
\[ \hat{\mu} = \frac{1}{n} \sum_{i=1}^n x_i \]
is not only unbiased but also the most efficient (minimum variance) estimator of the population mean.