Question

There are different ways to combine features of the Breusch-Pagan and White tests for heteroskedasticity. One possibility not covered in the text is to run the regression $$\widehat{u}_{i}^{2} \text { on } x_{i 1}, x_{i 2}, \dots, x_{i k}, \hat{y}_{i}^{2}, i=1, \dots, n$$ where the \(\hat{u}_{i}\) are the OLS residuals and the \(\hat{y}_{i}\) are the OLS fitted values. Then, we would test joint significance of \(x_{11}, x_{i 2}, \ldots, x_{i k}\) and \(\hat{y}_{i}^{2}\) (of course, we always include an intercept in this regression.) i. What are the \(d f\) associated with the proposed \(F\) test for heteroskedasticity? ii. Explain why the \(R\) -squared from the regression above will always be at least as large as the \(R\) squareds for the BP regression and the special case of the White test. iii. Does part (ii) imply that the new test always delivers a smaller \(p\) -value than either the BP or special case of the White statistic? Explain. iv. Suppose someone suggests also adding \(\hat{y}_{i}\) to the newly proposed test. What do you think of this idea?

Short answer

i. The degrees of freedom for the F-test is \(k+1\). ii. The R-squared will be at least as large due to extra terms. iii. A larger R-squared does not guarantee a smaller p-value. iv. Adding \(\hat{y}_i\) might introduce multicollinearity.

Step-by-step solution

Determine Degrees of Freedom

To find the degrees of freedom for the proposed F-test, consider the number of parameters being tested jointly for significance. You are testing the significance of the coefficients associated with the k original regressors \(x_{i1}, x_{i2}, \ldots, x_{ik}\) along with the \(\hat{y}_i^2\) term. Include the intercept in this setup, but it is not counted in the degrees of freedom for the F-test. Thus, the degrees of freedom for this test is \(k + 1\), where \(k\) is the number of explanatory variables.

Analyze R-squared Comparison

The R-squared value in a regression measures the proportion of variance explained by the model. Since the regression includes all variables from the BP test (which uses only the explanatory variables \(x\)) and the special case of the White test (which typically includes indicators, squares, and interactions), and adds an additional term \(\hat{y}_i^2\), its R-squared will be at least as large as those of the BP and White tests. This is due to the fact that adding more variables to a regression model usually increases, or maintains, the R-squared value.

Evaluate Implications for P-Value

The expectation might be that a higher R-squared leads to a lower p-value for the test. However, a higher R-squared doesn't necessarily imply a smaller p-value if multicollinearity is introduced or the added variable does not meaningfully contribute to the model. More variables can absorb degrees of freedom, thus not guaranteeing smaller p-values, as the F-statistic can be influenced both by the increase in explained variance and the loss of error degrees of freedom.

Consider Addition of Approximations

Adding \(\hat{y}_i\) to the regression may introduce multicollinearity due to the squared term \(\hat{y}_i^2\) already being included, potentially inflating standard errors and reducing the power of the test. It could disconnect the purpose of investigating heterogeneity specific to squared fitted values, by doubling up effects captured by \(\hat{y}_i^2\). Instead, the focus is better left on variables of interest to avoid redundancy and preserve test clarity.

Key concepts

Breusch-Pagan Test

The Breusch-Pagan test is a statistical method used to detect heteroskedasticity in a regression analysis. Heteroskedasticity occurs when the variance of errors is not constant across all levels of an independent variable, which can violate the assumptions of ordinary least squares (OLS) regression. This test aims to check for the presence of non-constant variance by examining the linear relationship between the squared residuals from the regression and the independent variables.
The procedure involves:

• First, estimating a regression model using OLS.
• Then, obtaining the residuals and squaring them.
• Lastly, performing a secondary regression of the squared residuals on the independent variables.

The test statistic, which follows an asymptotic chi-squared distribution, is calculated. A significant test statistic suggests evidence of heteroskedasticity, meaning that the variance changes at different levels of the independent variable.

White Test

The White test extends on the Breusch-Pagan test and is another common method to detect heteroskedasticity. Unlike the Breusch-Pagan test, the White test does not assume a specific functional form of heteroskedasticity. It is more flexible by including both squared terms of the independent variables and their pairwise interactions in the model.
To perform the White test, follow these steps:

• Estimate the initial regression and obtain the residuals.
• Regress the squared residuals on the independent variables along with their squares and interactions.

The resulting test statistic also follows a chi-squared distribution. The flexibility of the White test makes it suitable for detecting multiple forms of heteroskedasticity, including those unidentified by the Breusch-Pagan test. However, its complexity might increase the risk of overfitting, especially in models with many predictors.

Regression Analysis

Regression analysis is a statistical process for estimating the relationships among variables. It allows us to understand how the typical value of the dependent variable changes when one of the independent variables is varied, while the other independent variables remain fixed.
Here's an overview of key aspects:

• The process starts by identifying the dependent and independent variables.
• It involves fitting a model that best predicts the dependent variable from the independent variables, often using ordinary least squares (OLS) for linear models.
• Checking for underlying assumptions like linearity, homoscedasticity, and normality of errors is crucial for ensuring valid conclusions.

Regression analysis is widely used across numerous fields such as economics, biology, and engineering to make predictions and infer causal relationships.

Degrees of Freedom

Degrees of freedom (df) in statistics represent the number of independent values that can vary in an analysis without breaking any constraints. In regression analysis, the concept of degrees of freedom is essential for understanding how many samples are available to estimate parameters or assess the model's fit.
Specifically, in the context of testing for heteroskedasticity:

• The number of explanatory variables plus one (for the error term) gives the total parameters to estimate.
• Consequently, the model's residual degrees of freedom equal the number of observations minus the total number of parameters estimated including the intercept (N - k - 1).

The degrees of freedom are critical when interpreting test statistics and understanding the breadth of model flexibility.

R-squared Value

The R-squared value, also known as the coefficient of determination, measures the proportion of variance in the dependent variable that is predictable from the independent variables in a regression model. It provides an indication of the goodness of fit for the model.
An R-squared value ranges from 0 to 1:

• A value closer to 1 indicates that a larger portion of the dependent variable's variance is explained by the model.
• Conversely, a value near 0 suggests that the model does not explain much of the variance.

However, while a higher R-squared value can imply a better fit, it does not indicate that the model is valid or lacks bias. It is essential to consider R-squared alongside other diagnostics and statistical tests to ensure robust model inference.