Question

Consider again the conditions of Exercise 12, and let\(\theta = {c_1}{\beta _1} - {\beta _0}\), where \({c_1}\) is a constant. Determine an unbiased estimator \(\hat \theta \) of \(\theta \). For what value of \({c_1}\) will the M.S.E. of \(\hat \theta \) be smallest?

Short answer

The value of \(\hat \theta \) is \(\hat \theta = 0.548{c_1} - 40.892\) is an unbiased estimator of \(\theta \), and the value of \({c_1}\) which minimizes the MSE of \(\hat \theta \) is \(c_1^* = - 2.253\)

Step-by-step solution

Determine the value of unbiased estimator of \(\hat \theta \)

Define, \(\theta = - {\beta _0} + {c_1}{\beta _1}\), where \({\beta _0}\) and \({\beta _1}\) are the parameters of linear regression.

\({\hat \beta _0}\)and \({\hat \beta _1}\) are unbiased estimators of \({\beta _0}\) and \({\beta _1}\), respectively. Thus,

\(\begin{array}{l}E\left( {{{\hat \beta }_0}} \right) = {\beta _0}\\E\left( {{{\hat \beta }_1}} \right) = {\beta _1}\end{array}\)

Define their linear combination as,

\(\hat \theta = - {\hat \beta _0} + {c_1}{\hat \beta _1}\)

To show \(\hat \theta \) is unbiased for\(\theta \) as follows,

\(\begin{array}{c}E(\hat \theta ) = E\left( { - {{\hat \beta }_0} + {c_1}{{\hat \beta }_1}} \right)\\ = - E\left( {{{\hat \beta }_0}} \right) + {c_1}E\left( {{{\hat \beta }_1}} \right)\\ = - {\beta _0} + {c_1}{\beta _1}\\ = \theta \end{array}\)

From exercise 12, \({\hat \beta _0} = 40.892.\) and \({\hat \beta _1} = 0.548\).

Thus, the value of an unbiased estimate of \(\theta \) is,

\(\begin{array}{c}\hat \theta = - 40.892 + {c_1} \times 0.548\\ = 0.548{c_1} - 40.892\end{array}\)

The value of an unbiased estimate of \(\theta \) is \(0.548{c_1} - 40.892\).

Determine the M.S.E of \(\hat \theta \)

The MSE of an estimator \(\hat \theta \) can be written as,

\({\mathop{\rm MSE}\nolimits} (\hat \theta ) = {\mathop{\rm Var}\nolimits} (\hat \theta ) + {{\mathop{\rm Bias}\nolimits} ^2}\)

But the bias is 0.

Therefore,

\(\begin{array}{c}M.S.E(\hat \theta ) = Var(\hat \theta )\\ = Var\left( { - {{\hat \beta }_0} + {c_1}{{\hat \beta }_1}} \right)\\ = Var\left( {{{\hat \beta }_0}} \right) + Var\left( {{c_1}{{\hat \beta }_1}} \right) - 2Cov\left( {{{\hat \beta }_0},{c_1}{{\hat \beta }_1}} \right)\\ = Var\left( {{{\hat \beta }_0}} \right) + c_1^2Var\left( {{{\hat \beta }_1}} \right) - 2{c_1}Cov\left( {{{\hat \beta }_0},{{\hat \beta }_1}} \right)\end{array}\)

The variances of the estimated parameters \({\hat \beta _0}\) and \({\hat \beta _1}\) are,

\(\begin{array}{l}Var\left( {{{\hat \beta }_0}} \right) = {\sigma ^2}\left( {\frac{1}{n} + \frac{{{{\bar x}^2}}}{{s_x^2}}} \right)\\Var\left( {{{\hat \beta }_1}} \right) = \frac{{{\sigma ^2}}}{{s_x^2}}\end{array}\)

Also, the covariance of \({\hat \beta _0}\) and \({\hat \beta _1}\) is given as,

\(Cov\left( {{{\hat \beta }_0},{{\hat \beta }_1}} \right) = - \frac{{\bar x{\sigma ^2}}}{{s_x^2}}\)

From exercise 12 and 13, the following values are obtained,

\(\begin{array}{l}{\mathop{\rm Var}\nolimits} \left( {{{\hat \beta }_0}} \right) = 0.607{\sigma ^2}\quad \\{\mathop{\rm Var}\nolimits} \left( {{{\hat \beta }_1}} \right) = 0.095{\sigma ^2}\\{\mathop{\rm Cov}\nolimits} \left( {{{\hat \beta }_0},{{\hat \beta }_1}} \right) = - 0.214{\sigma ^2}\end{array}\)

Substituting the values,

\(\begin{array}{c}{{\mathop{\rm MSE}\nolimits} _{\hat \theta }}\left( {{c_1}} \right) = 0.607{\sigma ^2} + c_1^2\left( {0.095{\sigma ^2}} \right) + 2{c_1}\left( {0.214{\sigma ^2}} \right)\\ = \left( {0.095c_1^2 + 0.428{c_1} + 0.607} \right){\sigma ^2}\end{array}\)

Determine the value of \({{\bf{c}}_{\bf{1}}}\) which minimizes M.S.E of \(\hat \theta \)

To find the value of \({c_1}\) which minimizes the above expression.

Since \({\sigma ^2}\) is a constant, then it's equivalent to minimize the following function:

\(\begin{array}{c}{{\mathop{\rm MSE}\nolimits} _{\hat \theta }}\left( {{c_1}} \right) = f\left( {{c_1}} \right)\\ = 0.095c_1^2 + 0.428{c_1} + 0.607\end{array}\)

The first derivative of is,

\(f'\left( {{c_1}} \right) = 0.19{c_1} + 0.428\)

Equating the value equal to 0,

\(\begin{array}{c}{f^\prime }\left( {{c_1}} \right) = 0\\0.19{c_1} + 0.428 = 0\\{c_1} = - \frac{{0.428}}{{0.19}}\\ = - 2.253\end{array}\)

The second derivative of the function is,

\({f^{\prime \prime }}\left( {{c_1}} \right) = 0.19 > 0\)

which implies that \(f\) is strictly convex.

Thus, \({c_1} = - 2.253\) is the global minima for the function.

Therefore, \(c_1^* = - 2.253\)is the value which minimizes the MSE of \(\hat \theta \).