Question

Consider the following model \(5.81 y_{i}=\beta_{1}+\beta_{2} x_{i 2}+\beta_{3} x_{i 3}+\varepsilon_{i}, \quad i=1, \ldots, N\), where \(\left(y_{i}, x_{i 2}, x_{i 3}\right)\) are observed and have finite moments, and \(\varepsilon_{i}\) is an unobserved error term. Suppose this model is estimated by ordinary least squares. Denote the OLS estimator by \(b\). (a) What are the essential conditions required for unbiasedness of \(b\) ? What are the essential conditions required for consistency of \(b\) ? Explain the difference between unbiasedness and consistency. (b) Show how the conditions for consistency can be written as moment conditions (if you have not done so already). Explain how a method of moments estimator can be derived from these moment conditions. Is the resulting estimator any different from the OLS one? Now suppose that \(\operatorname{cov}\left\\{\varepsilon_{i}, x_{i 3}\right\\} \neq 0\) (c) Give two examples of cases where one can expect a nonzero correlation between a regressor, \(x_{i 3}\), and the error \(\varepsilon_{i}\). (d) In this case, is it possible still to make appropriate inferences based on the OLS estimator while adjusting the standard errors appropriately? (e) Explain how an instrumental variable, \(z_{i}\), say, leads to a new moment condition and, consequently, an alternative estimator for \(\beta .\) (f) Why does this alternative estimator lead to a smaller \(R^{2}\) than the OLS one? What does this say of the \(R^{2}\) as a measure for the adequacy of the model? (g) Why can we not choose \(z_{i}=x_{i 2}\) as an instrument for \(x_{i 3}\), even if \(E\left\\{x_{i 2} \varepsilon_{i}\right\\}=0\) ? Would it be possible to use \(x_{i 2}^{2}\) as an instrument for \(x_{i 3} ?\)

Short answer

Unbiasedness requires zero mean errors. Consistency requires errors to vanish as sample size grows. Methods of moments and IV solve endogeneity. Smaller \( R^2 \) in IV shows bias correction at variance cost.

Step-by-step solution

Conditions for Unbiasedness and Consistency

Unbiasedness requires that the expected value of the errors is zero and that the regressors are uncorrelated with the error term. Mathematically, \[ E[u_i | X_i] = 0 \] for all i. For consistency, the limit distribution as the sample size goes to infinity is needed: \[ \text{plim}(b) = \beta \] This requires the error terms to be weakly dependent and the regressors to have finite fourth moments.

Difference between Unbiasedness and Consistency

Unbiasedness means the expected value of the estimator equals the true parameter value, regardless of sample size. Consistency means the estimator converges in probability to the true parameter value as the sample size approaches infinity.

Moment Conditions for Consistency

The conditions for consistency can be written as: \[ E[X_i(u_i)] = 0 \] Method of moments uses these conditions to solve for the parameters by equating sample moments to population moments, leading to: \[ \frac{1}{N}\textstyle\boldsymbol{X}'\boldsymbol{e} \rightarrow 0 \]

Method of Moments Estimator

To derive the method of moments estimator, set the sample moments equal to zero: \[ \frac{1}{N} \textstyle\boldsymbol{X}'(\textbf{y} - \boldsymbol{X}\beta) = 0 \] Solving for \( \beta \) gives: \[ \beta = (\boldsymbol{X}'\boldsymbol{X})^{-1}\boldsymbol{X}'\textbf{y} \] which is identical to the OLS estimator.

Nonzero Correlation Between \( x_{i3} \) and \( u_i \)

Examples of nonzero correlations include: 1. Omitted variable bias where the omitted variable affects both \( x_{i3} \) and \( u_i \). 2. Measurement error in \( x_{i3} \).

Adjusting Standard Errors

If \( \text{cov}(u_i, x_{i3}) eq 0 \), OLS provides biased and inconsistent estimates. Adjusting standard errors does not fix the bias; instrumental variable (IV) methods are required.

Instrumental Variables and Moment Conditions

An IV \( z_i \) is used with the moment condition: \[ E[z_i (u_i)] = 0 \] The new estimator is obtained by: \[ \beta = (Z'X)^{-1}Z'Y \] This estimator is different from OLS.

Smaller \( R^2 \) in IV Estimator

IV estimators usually lead to a smaller \( R^2 \) because IV addresses bias at the cost of higher variance. This suggests \( R^2 \) is not a reliable measure of model adequacy when endogeneity is present.

Selection of Instruments

Using \( x_{i2} \) as an instrument for \( x_{i3} \) is invalid because it doesn't satisfy exclusion restrictions if \( x_{i2} \) and \( u_i \) are correlated. However, \( x_{i2}^2 \) could be valid if it correlates with \( x_{i3} \) but not \( u_i \).

Key concepts

Unbiasedness Conditions

To ensure that the OLS estimator, denoted as \(b\), is unbiased, several conditions must be met. The primary condition is that the expected value of the error term \( \varepsilon_i \) must be zero, given the values of the regressors \( x_{i2} \) and \( x_{i3} \). Mathematically, this can be expressed as: \[ E[ \varepsilon_i | X_i ] = 0 \] for all i.
Additionally, the regressors should not be correlated with the error term. In other words, \( \text{cov}( X_i, \varepsilon_i ) = 0 \).
Together, these conditions ensure that the OLS estimator accurately reflects the true parameters, without being systematically biased. In absence of these conditions, the estimator could consistently overestimate or underestimate the true parameter values.

Consistency in Estimators

Consistency is about the behavior of an estimator as the sample size increases indefinitely. For the OLS estimator to be consistent, it must converge to the true parameter value as the sample size approaches infinity. Mathematically, this is shown as:
\[ \text{plim}( b ) = \beta \]
Consistency requires that the error terms be *weakly dependent* and that the regressors have finite moments. This means that as the number of observations grows, the distribution of the estimator narrows around the true parameter value. Consistency ensures accurate inference in large samples and is crucial for reliable estimations.

Method of Moments

The *Method of Moments* is a technique used to derive estimators based on sample moments. To derive a method of moments estimator, we start by writing the moment conditions required for consistency: \[ E[ X_i \( \varepsilon_i \) ] = 0 \]
This indicates that the sample moments should approximate the population moments.

We then solve the following equation to obtain the estimator:
\[ \frac{1}{N} \boldsymbol{X}'( \textbf{y} - \boldsymbol{X} \beta ) = 0 \]
Solving this gives us:
\[ \beta = ( \boldsymbol{X}' \boldsymbol{X} )^{-1} \boldsymbol{X}' \textbf{y} \]
Notice that this is identical to the OLS estimator. Therefore, while the method of moments provides a different perspective, it leads to the same result as OLS in this context.

Instrumental Variables

Instrumental Variables (IV) are used to address the problem of endogeneity. Endogeneity occurs when an explanatory variable is correlated with the error term, leading to biased and inconsistent OLS estimates.
To correct for this, we introduce an instrument, \( z_i \), which must satisfy two conditions:

• The instrument must be correlated with the endogenous regressor, \( x_{i3} \).
• The instrument must be uncorrelated with the error term, \( \varepsilon_i \).

The key moment condition now is:
\[ E[ z_i ( \varepsilon_i ) ] = 0 \]
From this moment condition, an alternative estimator can be derived as:
\[ \beta = ( Z'X )^{-1} Z'Y \]
This IV estimator corrects for the bias introduced by the endogeneity of \( x_{i3} \).

Endogeneity

Endogeneity is a situation in which an explanatory variable is correlated with the error term in a regression model. This can arise due to various reasons, such as:

• **Omitted Variable Bias**: An important variable that affects both the dependent variable and an independent variable is not included in the model.
• **Measurement Error**: Inaccurate measurement of a variable can lead to correlation with the error term.

When endogeneity is present, it violates one of the key conditions for the OLS estimator being unbiased and consistent. It often leads to biased estimates that are not reliable. In such cases, techniques like Instrumental Variables (IV) are used to obtain consistent estimates.