Question

i. Consider the simple regression model \(y=\beta_{0}+\beta_{1} x+u\) under the first four Gauss-Markov assumptions. For some function \(g(x),\) for example \(g(x)=x^{2}\) or \(g(x)=\log \left(1+x^{2}\right),\) define \(z_{i}=g\left(x_{i}\right) .\) Define a slope estimator as $$ \tilde{\beta}_{1}=\left(\sum_{i=1}^{n}\left(z_{i}-\bar{z}\right) y_{i}\right) /\left(\sum_{i=1}^{n}\left(z_{i}-\bar{z}\right) x_{i}\right) $$ Show that \(\bar{\beta}_{1}\) is linear and unbiased. Remember, because \(\mathrm{E}(u | x)=0,\) you can treat both \(x_{i}\) and \(z_{i}\) as nonrandom in your derivation. ii. Add the homoskedasticity assumption, MLR.5. Show that $$ \operatorname{Var}\left(\tilde{\beta}_{1}\right)=\sigma^{2}\left(\sum_{i=1}^{n}\left(z_{i}-\bar{z}\right)^{2}\right) /\left(\sum_{i=1}^{n}\left(z_{i}-\bar{z}\right) x_{i}\right)^{2} $$ iii. Show directly that, under the Gauss-Markov assumptions, Var( \(\left.\hat{\beta}_{1}\right) \leq \operatorname{Var}\left(\tilde{\beta}_{1}\right),\) where \(\hat{\beta}_{1}\) is the OLS estimator. [Hint: The Cauchy-Schwartz inequality in Appendix B implies that $$ \left(n^{-1} \sum_{i=1}^{n}\left(z_{i}-\bar{z}\right)\left(x_{i}-\bar{x}\right)\right)^{2} \leq\left(n^{-1} \sum_{i=1}^{n}\left(z_{i}-\bar{z}\right)^{2}\right)\left(n^{-1} \sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2}\right) $$ notice that we can drop \(\bar{x}\) from the sample covariance.

Short answer

\(\tilde{\beta}_1\) is unbiased, but its variance is generally larger than \(\hat{\beta}_1\) due to Cauchy-Schwarz, under assumptions.

Step-by-step solution

Understanding the Slope Estimator

We have the slope estimator \(\tilde{\beta}_{1}\) defined as \(\left(\sum_{i=1}^{n}\left(z_{i}-\bar{z}\right) y_{i}\right) /\left(\sum_{i=1}^{n}\left(z_{i}-\bar{z}\right) x_{i}\right)\). Our task is to show it is linear and unbiased under the Gauss-Markov assumptions.

Verify Linearity and Unbiasedness

To verify linearity and unbiasedness, express \(\tilde{\beta}_1\) as a function of \(y_i = \beta_0 + \beta_1 x_i + u_i\). Substituting \(y_i\), we have:\[\tilde{\beta}_1 = \frac{\sum_{i=1}^{n} (z_i - \bar{z})(\beta_{0} + \beta_{1} x_i + u_i)}{\sum_{i=1}^{n}(z_i - \bar{z}) x_i}\]Since the sum of errors \(u_i\) weighted by \(z_i - \bar{z}\) is zero by assumption, \(\operatorname{E}(u|x) = 0\), \(\tilde{\beta}_1\) is an unbiased estimator of \(\beta_1\).

Introduce Homoscedasticity

With MLR.5 (Homoscedasticity) added, we find the variance of \(\tilde{\beta}_1\). Substitute \(\operatorname{Var}(u_i) = \sigma^2\) and use:\[\operatorname{Var}\left(\tilde{\beta}_1\right) = \sigma^2 \sum_{i=1}^{n}(z_i - \bar{z})^2 / \left[\sum_{i=1}^{n}(z_i - \bar{z}) x_i\right]^2\]

Cauchy-Schwarz Inequality Application

To prove \(\operatorname{Var}(\hat{\beta}_1) \leq \operatorname{Var}(\tilde{\beta}_1)\), apply the Cauchy-Schwarz inequality, which states:\[\left(n^{-1} \sum_{i=1}^{n} (z_i - \bar{z}) (x_i - \bar{x})\right)^2 \leq \left(n^{-1} \sum_{i=1}^{n} (z_i - \bar{z})^2\right) \left(n^{-1} \sum_{i=1}^{n} (x_i - \bar{x})^2\right)\]This inequality ensures the denominator of the variance of \(\hat{\beta}_1\) is always larger, hence, the variance is smaller or equal compared to \(\tilde{\beta}_1\).

Conclusion

Under the Gauss-Markov assumptions, particularly with the inclusion of homoscedasticity, \(\tilde{\beta}_1\) is a linear, unbiased estimator but not necessarily the most efficient (minimum variance); the OLS \(\hat{\beta}_1\) typically has a smaller variance.

Key concepts

Slope Estimator

The slope estimator is a way to measure the relationship between two variables in a regression model. Specifically, it represents how changes in an independent variable (\(x\)) influence changes in a dependent variable (\(y\)). In the given exercise, the slope estimator \(\tilde{\beta}_{1}\) is defined for a function \(g(x)\). This estimator is made linear by expressing it as a weighted sum of the observed data and the error term \(u\).

To ensure unbiasedness, we derive \(\tilde{\beta}_{1}\) from substituting \(y_i = \beta_0 + \beta_1 x_i + u_i\) into the estimator formula. Because the average of the errors weighted by \(z_i - \bar{z}\) equals zero, \(\tilde{\beta}_{1}\) becomes an unbiased estimator of \(\beta_1\). This unbiased nature implies that, on average, our estimator will hit the true parameter without systematic error.

Homoscedasticity

Homoscedasticity is a crucial assumption in regression models, particularly in the context of the Gauss-Markov theorem. It means that the variance of the error term \(u_i\) is constant across all levels of the independent variable \(x_i\). This uniformity of variance is crucial in ensuring the precision and reliability of our estimators.

With homoscedasticity, the variance of the slope estimator \(\tilde{\beta}_1\) can be calculated as given. The assumption allows us to use \(\operatorname{Var}(u_i) = \sigma^2\) in the variance formula. A consistent error variance helps in deriving efficient estimates, meaning that our estimation of \(\beta_1\) is not only unbiased but also has a minimized variance which brings us closer to the true value of \(\beta_1\). When this assumption holds, estimators will generally provide accurate predictions.

Cauchy-Schwarz Inequality

The Cauchy-Schwarz inequality is a mathematical tool used to compare estimates effectively in statistics. In regression analysis, this inequality provides a way to compare variances and establish that one is smaller or equal to another. In this exercise, it helps to demonstrate that the variance of the OLS estimator \(\hat{\beta}_1\) is always less than or equal to that of \(\tilde{\beta}_1\).

The inequality is expressed in the setup:

• \(\left(n^{-1} \sum_{i=1}^{n} (z_i - \bar{z}) (x_i - \bar{x})\right)^2 \leq \left(n^{-1} \sum_{i=1}^{n} (z_i - \bar{z})^2\right) \left(n^{-1} \sum_{i=1}^{n} (x_i - \bar{x})^2\right)\)

This means that the product of the variances of standardized variables provides an upper bound to their joint covariance calculated in a similar standardized form. It's vital because it underpins the efficiency of the OLS estimator, showing that it is a more reliable estimator than \(\tilde{\beta}_1\) from the standpoint of variance.

OLS Estimator

The Ordinary Least Squares (OLS) estimator, represented by \(\hat{\beta}_{1}\), is a common method for estimating the parameters in linear regression models. It works by minimizing the sum of the squared differences between the observed values and the values predicted by the linear model. The efficiency of this estimator is largely due to the properties explored under the Gauss-Markov theorem.

The OLS estimator is particularly valued because it is the Best Linear Unbiased Estimator (BLUE) under the Gauss-Markov assumptions, which include linearity, unbiasedness, and efficiency. This means OLS provides the estimates of \(\beta_1\) with the smallest possible variance among all unbiased linear estimators. This superiority is partly proved using the Cauchy-Schwarz inequality, as it typically results in the smallest variance, allowing us to be more confident in its predictions relative to other estimators like \(\tilde{\beta}_1\).

In essence, the OLS estimator creates a balance between bias and variance, making it a cornerstone technique in statistical modeling.