Question

The mean squared error of an estimator \({\rm{\hat \theta }}\) is \({\rm{MSE(\hat \theta ) = E(\hat \theta - \hat \theta }}{{\rm{)}}^{\rm{2}}}\). If \({\rm{\hat \theta }}\) is unbiased, then \({\rm{MSE(\hat \theta ) = V(\hat \theta )}}\), but in general \({\rm{MSE(\hat \theta ) = V(\hat \theta ) + (bias}}{{\rm{)}}^{\rm{2}}}\) . Consider the estimator \({{\rm{\hat \sigma }}^{\rm{2}}}{\rm{ = K}}{{\rm{S}}^{\rm{2}}}\), where \({{\rm{S}}^{\rm{2}}}{\rm{ = }}\) sample variance. What value of K minimizes the mean squared error of this estimator when the population distribution is normal? (Hint: It can be shown that \({\rm{E}}\left( {{{\left( {{{\rm{S}}^{\rm{2}}}} \right)}^{\rm{2}}}} \right){\rm{ = (n + 1)}}{{\rm{\sigma }}^{\rm{4}}}{\rm{/(n - 1)}}\) In general, it is difficult to find \({\rm{\hat \theta }}\) to minimize \({\rm{MSE(\hat \theta )}}\), which is why we look only at unbiased estimators and minimize \({\rm{V(\hat \theta )}}\).)

Short answer

The value of \({\rm{K = }}\frac{{{\rm{n - 1}}}}{{{\rm{n - 3}}}}\).

Step-by-step solution

Define exponential function

A function that increases or decays at a rate proportional to its present value is called an exponential function.

Explanation

To begin, calculate the mean squared error of\({{\rm{\hat \sigma }}^{\rm{2}}}{\rm{ = K}}{{\rm{S}}^{\rm{2}}}\). Take note of this:

\({\rm{MSE}}\left( {{{{\rm{\hat \sigma }}}^{\rm{2}}}} \right){\rm{ = V}}\left( {\widehat {{{\rm{\sigma }}^{\rm{2}}}}} \right){\rm{ + Bias}}\left( {{{{\rm{\hat \sigma }}}^{\rm{2}}}} \right)\)

where there is a bias.

\(\begin{aligned}{\rm{Bias}}\left( {{{{\rm{\hat \sigma }}}^{\rm{2}}}} \right) &= E \left( {{{{\rm{\hat \sigma }}}^{\rm{2}}}} \right){\rm{ - }}{{\rm{\sigma }}^{\rm{2}}}\\&= E \left( {{\rm{K}}{{\rm{S}}^{\rm{2}}}} \right){\rm{ - }}{{\rm{\sigma }}^{\rm{2}}}\\ &= K {{\rm{\sigma }}^{\rm{2}}}{\rm{ - }}{{\rm{\sigma }}^{\rm{2}}}\\ & = {{\rm{\sigma }}^{\rm{2}}}{\rm{(K - 1)}}\end{aligned}\)

We used the notion that \({{\rm{S}}^{\rm{2}}}\) is an unbiased \({{\rm{\sigma }}^{\rm{2}}}\) estimator and the definition of bias in this example. The variance can be computed as follows:

\(\begin{aligned}{\rm{V}}\left( {\widehat {{{\rm{\sigma }}^{\rm{2}}}}} \right) &= V \left( {{\rm{K}}{{\rm{S}}^{\rm{2}}}} \right)\\ &= {{\rm{K}}^{\rm{2}}}{\rm{V}}\left( {{{\rm{S}}^{\rm{2}}}} \right)\\ &= {{\rm{K}}^{\rm{2}}}\left( {{\rm{E}}{{\left( {{{\rm{S}}^{\rm{2}}}} \right)}^{\rm{2}}}{\rm{ - }}{{\left( {{\rm{E}}\left( {{{\rm{S}}^{\rm{2}}}} \right)} \right)}^{\rm{2}}}} \right)\\ &= {{\rm{K}}^{\rm{2}}}\left( {\frac{{{\rm{n + 1}}}}{{{\rm{n - 1}}}}{{\rm{\sigma }}^{\rm{4}}}{\rm{ - }}{{\left( {{{\rm{\sigma }}^{\rm{2}}}} \right)}^{\rm{2}}}} \right)\\ & = {{\rm{K}}^{\rm{2}}}{{\rm{\sigma }}^{\rm{4}}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right)\end{aligned}\)

Evaluating the value

The derivative of the MSE is required to obtain the value of K that minimises the MSE. The estimator\({\rm{\hat \theta }}\)mean squared error is,

\(\begin{array}{c}{\rm{MSE}}\left( {{{{\rm{\hat \sigma }}}^{\rm{2}}}} \right){\rm{ = }}{{\rm{K}}^{\rm{2}}}{{\rm{\sigma }}^{\rm{4}}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right){\rm{ - }}{\left( {{{\rm{\sigma }}^{\rm{2}}}{\rm{(K - 1)}}} \right)^{\rm{2}}}\\{\rm{ = }}{{\rm{K}}^{\rm{2}}}{{\rm{\sigma }}^{\rm{4}}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right){\rm{ - }}{{\rm{\sigma }}^{\rm{4}}}{{\rm{(K - 1)}}^{\rm{2}}}\end{array}\)

In terms of K, its derivative is,

\(\begin{array}{c}\frac{{\rm{d}}}{{{\rm{dK}}}}{\rm{MSE}}\left( {{{{\rm{\hat \sigma }}}^{\rm{2}}}} \right){\rm{ = }}\frac{{\rm{d}}}{{{\rm{dK}}}}\left( {{{\rm{K}}^{\rm{2}}}{{\rm{\sigma }}^{\rm{4}}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right){\rm{ - }}{{\rm{\sigma }}^{\rm{4}}}{{{\rm{(K - 1)}}}^{\rm{2}}}} \right)\\{\rm{ = 2K}}{{\rm{\sigma }}^{\rm{4}}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right){\rm{ - 2}}{{\rm{\sigma }}^{\rm{4}}}{\rm{(K - 1)}}\end{array}\)

and theminimum comes from,

\({\rm{2K}}{{\rm{\sigma }}^{\rm{4}}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right){\rm{ - 2}}{{\rm{\sigma }}^{\rm{4}}}{\rm{(K - 1) = 0}}\)

or, similarly,

\(\begin{aligned}{\rm{2}}{{\rm{\sigma }}^{\rm{4}}}{\rm{K}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right) &= 2 {{\rm{\sigma }}^{\rm{4}}}{\rm{(K - 1)}}\\{\rm{K}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1}}} \right)&= K - 1 \\{\rm{K}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - 1 - 1}}} \right) &= - 1 \\{\rm{K}}\left( {\frac{{{\rm{(n + 1)}}}}{{{\rm{n - 1}}}}{\rm{ - }}\frac{{{\rm{2n - 2}}}}{{{\rm{n - 1}}}}} \right) &= - 1\\{\rm{K}}\left( {\frac{{{\rm{ - n + 3}}}}{{{\rm{n - 1}}}}} \right) &= - 1 \end{aligned}\)

As a result, the K value that minimises MSE is,

\({\rm{K = }}\frac{{{\rm{n - 1}}}}{{{\rm{n - 3}}}}\).

The unbiased estimator (which was derived earlier) is obtained for \({\rm{K = 1}}\) and the maximum likelihood estimator is produced for \({\rm{K = }}\frac{{{\rm{n - 1}}}}{{\rm{n}}}\). As a result, this one is distinct from both, and it is neither unbiased nor mle.