Question

Consider the following linear regression model: \(y_{i}=\beta_{1}+\beta_{2} x_{i 2}+\beta_{3} x_{i 3}+\varepsilon_{i}=x_{i}^{\prime} \beta+\varepsilon_{i} .\) (a) Explain how the ordinary least squares estimator for \(\beta\) is determined and derive an expression for \(b\). (b) Which assumptions are needed to make \(b\) an unbiased estimator for \(\beta\) ? (c) Explain how a confidence interval for \(\beta_{2}\) can be constructed. Which additional assumptions are needed? (d) Explain how one can test the hypothesis that \(\beta_{3}=1\). (e) Explain how one can test the hypothesis that \(\beta_{2}+\beta_{3}=0\). (f) Explain how one can test the hypothesis that \(\beta_{2}=\beta_{3}=0\). (g) Which assumptions are needed to make \(b\) a consistent estimator for \(\beta\) ? (h) Suppose that \(x_{i 2}=2+3 x_{i 3}\). What will happen if you try to estimate the above model? (i) Suppose that the model is estimated with \(x_{i 2}^{*}=2 x_{i 2}-2\) included rather than \(x_{i 2}\). How are the coefficients in this model related to those in the original model? And the \(R^{2}\) s? (j) Suppose that \(x_{i 2}=x_{i 3}+u_{i}\), where \(u_{i}\) and \(x_{i 3}\) are uncorrelated. Suppose that the model is estimated with \(u_{i}\) included rather than \(x_{i 2}\). How are the coefficients in this model related to those in the original model? And the \(R^{2} \mathrm{~s}\) ?

Short answer

The OLS estimator \(b = (X'X)^{-1} X'y\) minimizes the sum of squared residuals. For \(b\) to be unbiased, linearity, exogeneity, and absence of perfect multicollinearity assumptions must hold. Confidence intervals and hypothesis tests involve standard errors and test statistics under various assumptions.

Step-by-step solution

- Ordinary Least Squares Estimator

The ordinary least squares (OLS) estimator for \(\beta\) is determined by minimizing the sum of squared residuals \(\text{SSR} = \text{min}_\beta \sum_{i=1}^n (y_i - x_i' \beta)^2\). The solution for \(\beta\) is obtained by differentiating the SSR with respect to \(\beta\) and setting the derivative to zero: \(\frac{\partial \text{SSR}}{\partial \beta} = -2 \sum_{i=1}^n x_i (y_i - x_i' \beta) = 0\). This leads to the normal equations: \(X'X \beta = X'y\). Solving for \(\beta\), we get the OLS estimator: \(b = (X'X)^{-1} X'y\).

- Unbiased Estimator

For \(b\) to be an unbiased estimator for \(\beta\), the following assumptions need to hold: (1) Linearity: The relationship between the independent variables and the dependent variable is linear. (2) Exogeneity: The error terms \(\varepsilon_i\) have an expected value of zero given \(x_i\). (3) No perfect multicollinearity: The independent variables are not perfectly correlated. Under these assumptions, \(\text{E}(b) = \beta\).

- Confidence Interval for \(\beta_2\)

A \(100(1-\alpha)%\) confidence interval for \(\beta_2\) can be constructed using the formula: \(\hat{\beta_2} \pm t_{\alpha/2, n-k} \cdot \text{SE}(\hat{\beta_2})\). The additional assumptions needed are that the errors \(\varepsilon_i\) are normally distributed and homoscedastic (constant variance).

- Testing \(\beta_3 = 1\)

To test the hypothesis \(H_0: \beta_3 = 1\), use the t-test. Calculate the test statistic: \(t = \frac{\hat{\beta_3} - 1}{\text{SE}(\hat{\beta_3})}\). Compare the calculated t-value with the critical t-value from the t-distribution with \(n-k\) degrees of freedom.

- Testing \(\beta_2 + \beta_3 = 0\)

To test the hypothesis \(H_0: \beta_2 + \beta_3 = 0\), form a new variable \(z_i = x_{i2} + x_{i3}\) and include it in the regression. Apply the t-test to \(\hat{\beta_z}\) (the coefficient of \(z_i\)): \(t = \frac{\hat{\beta_z}}{\text{SE}(\hat{\beta_z})}\).

- Testing \(\beta_2 = \beta_3 = 0\)

To test the joint hypothesis \(H_0: \beta_2 = \beta_3 = 0\), use the F-test. Calculate the F-statistic: \(F = \frac{(SSR_{restricted} - SSR_{unrestricted}) / q}{SSR_{unrestricted} / (n - k)}\), where \(q\) is the number of restrictions. Compare the calculated F-value with the critical F-value.

- Consistency of Estimator

For \(b\) to be a consistent estimator for \(\beta\), the following assumptions are required: (1) The exogeneity assumption (the error term has zero conditional mean). (2) The variation in the independent variables increases with the sample size. Under these assumptions, \(b\) converges in probability to \(\beta\).

- Multicollinearity Issue

If \(x_{i2} = 2 + 3 x_{i3}\), there is perfect multicollinearity, leading to an inability to estimate the model because \(X'X\) is not invertible. Hence, OLS fails.

- Coefficients with Transformed Variable

If the model is estimated with \(x_{i2}^* = 2 x_{i2} - 2\) instead of \(x_{i2}\), the coefficients adjust accordingly. Let \(\beta_2^* = \beta_2/2\). The coefficients are related as \(\beta_2^* = \beta_2/2\) and \(R^2\) remains unchanged as it measures the proportion of variability explained by the model.

- Model with Uncorrelated Error \(u_{i}\)

If \(x_{i2} = x_{i3} + u_i\), and \(u_i\) and \(x_{i3}\) are uncorrelated, estimating the model with \(u_i\) instead of \(x_{i2}\) alters coefficients. Coefficients remain consistent but may have higher standard errors. Adjusted \(R^2\) stays similar as explained variance depends on unchanged \(x_{i3}\).

Key concepts

ordinary least squares estimator

The Ordinary Least Squares (OLS) estimator is a method used to estimate the coefficients of a linear regression model. It works by minimizing the sum of the squared differences between the observed values and the values predicted by the model. In math terms, it minimizes the sum of squared residuals (SSR): \[\text{SSR} = \min_{\beta} \sum_{i=1}^n (y_i - x_i' \beta)^2.\]The solution can be found by setting the derivative with respect to \(\beta\) to zero, leading to: \[\frac{\partial \text{SSR}}{\partial \beta} = -2 \sum_{i=1}^n x_i (y_i - x_i' \beta) = 0.\]This results in the normal equations: \[X'X \beta = X'y.\]By solving for \(\beta\), we obtain the OLS estimator: \[b = (X'X)^{-1}X'y.\]

unbiased estimator

An estimator is considered unbiased if, on average, it hits the true population parameter. For the OLS estimator \(b\) to be unbiased for \(\beta\), certain assumptions must be met:

1. **Linearity**: The relationship between the independent and dependent variables is linear.
2. **Exogeneity**: The error terms have an expected value of zero given the predictors, i.e., \(E(\varepsilon_i | x_i) = 0\).
3. **No perfect multicollinearity**: The independent variables are not perfectly correlated.

Under these assumptions, the expectation of \(b\) is the true parameter \(\beta\):
\(E(b) = \beta\).

confidence interval

A confidence interval offers a range of values within which the true parameter value is expected to fall. To construct a confidence interval for \(\beta_2\):

1. Calculate the estimate \(\hat{\beta_2}\).
2. Determine the standard error of \(\hat{\beta_2}\) (SE(\(\hat{\beta_2}\))).
3. Use the critical value from the t-distribution (\(t_{\alpha/2, n-k}\)).

The formula is:
\(\hat{\beta_2} \pm t_{\alpha/2, n-k} \cdot \text{SE}(\hat{\beta_2})\).
Additional assumptions needed include normally distributed errors and homoscedasticity (errors with constant variance).

hypothesis testing

Hypothesis testing involves determining if certain statements about the parameters of your model are true. For example:

**Testing \(\beta_3 = 1\):** Use the t-test. Compute the test statistic:
\(t = \frac{\hat{\beta_3} - 1}{\text{SE}(\hat{\beta_3})}\).
Compare this t-value with the critical t-value from the t-distribution.

**Testing \(\beta_2 + \beta_3 = 0\):** Form a new variable \(z_i = x_{i2} + x_{i3}\) and include it in the regression. Apply the t-test to \(\hat{\beta_z}\):
\(t = \frac{\hat{\beta_z}}{\text{SE}(\hat{\beta_z})}\).

**Testing \(\beta_2 = \beta_3 = 0\):** Use the F-test. Calculate the F-statistic:
\(F = \frac{(\text{SSR}_{restricted} - \text{SSR}_{unrestricted}) / q}{\text{SSR}_{unrestricted} / (n - k)}\), where \(q\) is the number of restrictions.

multicollinearity

Multicollinearity occurs when the independent variables in a regression model are highly correlated. This can lead to problems such as:

1. **Unreliable Estimates**: Coefficients may become sensitive to small changes in the data.
2. **Inflated Standard Errors**: Making it harder to reject the null hypothesis.

For instance, if \(x_{i2} = 2 + 3 x_{i3}\), there is perfect multicollinearity. This makes the matrix \(X'X\) non-invertible, causing OLS to fail.

consistent estimator

An estimator is consistent if it converges in probability to the true parameter value as the sample size increases. For \(b\) to be a consistent estimator of \(\beta\), these conditions must be met:

1. **Exogeneity**: The error term has a zero conditional mean.
2. **Increasing Variation**: As the sample size grows, the variation in the independent variables must increase.

Under these assumptions, \(b\) converges to \(\beta\) as the sample size \(n\) approaches infinity.

t-test

The t-test is used to determine if a specific coefficient equals a given value (often zero). Let's say we want to test \(\beta_3 = 1\):

1. **Calculate the estimator** \(\hat{\beta_3}\).
2. **Determine the standard error** of \(\hat{\beta_3}\).
3. **Compute the t-statistic**:
\(t = \frac{\hat{\beta_3} - 1}{\text{SE}(\hat{\beta_3})}\).
4. **Compare** the t-value to the critical value from the t-distribution with \(n-k\) degrees of freedom.

If the computed t-value exceeds the critical value, we reject the null hypothesis.

F-test

The F-test is used to test the joint hypothesis that multiple coefficients equal zero, such as testing \(\beta_2 = \beta_3 = 0\). Steps include:

1. **Estimate both restricted and unrestricted models**:
\(\text{SSR}_{restricted}\) and \(\text{SSR}_{unrestricted}\).
2. **Calculate the F-statistic**:
\(F = \frac{(\text{SSR}_{restricted} - \text{SSR}_{unrestricted}) / q}{\text{SSR}_{unrestricted} / (n - k)}\), where \(q\) is the number of restrictions.
3. **Compare** the F-value with the critical F-value from the F-distribution.

If the F-value exceeds the critical value, we reject the null hypothesis.