Question

The article "The Caseload Controversy and the Study of Criminal Courts" (Journal of Criminal Law and Criminology [1979]: 89-101) used a multiple regression analysis to help assess the impact of judicial caseload on the processing of criminal court cases. Data were collected in the Chicago criminal courts on the following variables: $$ \begin{aligned} y &=\text { number of indictments } \\ x_{1} &=\text { number of cases on the docket } \end{aligned} $$ \(x_{2}=\) number of cases pending in criminal court trial system The estimated regression equation (based on \(n=367\) observations) was $$ \hat{y}=28-.05 x_{1}-.003 x_{2}+.00002 x_{3} $$ where \(x_{3}=x_{1} x_{2}\) a. The reported value of \(R^{2}\) was . 16. Conduct the model utility test. Use a \(.05\) significance level. b. Given the results of the test in Part (a), does it surprise you that the \(R^{2}\) value is so low? Can you think of a possible explanation for this? c. How does adjusted \(R^{2}\) compare to \(R^{2}\) ?

Short answer

The model utility test verifies whether the regression model is useful based on its F statistic and comparing with the F-critical value. The low \(R^{2}\) value could suggest that the predictors are weakly related to the dependent variable or missing important predictors. The comparison of \(R^{2}\) and adjusted \(R^{2}\) examines the contribution of predictors. While the former always increases when more predictors are included, the latter will decrease if new predictors don't significantly improve the model.

Step-by-step solution

model utility test

In order to conduct the model utility test, we must check if the Regression Model would be useful to predict the response. In multiple regression, the null hypothesis \(H_{0}\): All regression coefficients are equal to zero. And the alternative hypothesis \(H_{a}\): At least one regression coefficient is not zero. Since we are given an \(R^{2}\) of 0.16 and a significance level of 0.05, we can calculate the F statistic using the formula \(F = R^{2}/(1-R^{2}) * (n-p-1)/p\), where n is the number of observations and p is the number of predictors. Then, we must check if the calculated F-value is greater than the F-critical value from the F-distribution table for the given significance level. If it's greater then we reject the null hypothesis suggesting our model is useful.

Interpret the result from Step 1

Based on the result of the F-test in Step 1, interpret the outcome. If you reject the null hypothesis, it means that at least one predictor variable's coefficient is not zero, which suggests that the model has some predictive power. If we fail to reject the null hypothesis, it indicates that the model has no predictive power.

Discuss the \(R^{2}\) value

After the model utility test, discuss why the \(R^{2}\) value, which represents the proportion of the variance for a dependent variable that's explained by an independent variable(s), could be low. This could be due to a weak relationship between predictors and the dependent variable, or that important predictors are missing.

Compare \(R^{2}\) and adjusted \(R^{2}\)

Adjusted \(R^{2}\) takes into account the number of predictors in the model, adjusting for the increase of \(R^{2}\) when additional predictors are included. While \(R^{2}\) always increases as more predictors are added, adjusted \(R^{2}\) could decrease if the addition of the predictor doesn't significantly improve the model. When comparing these two, if both values are close it means the predictors all contribute to the model. Conversely, if the adjusted \(R^{2}\) is much lower than \(R^{2}\), some predictors may not be contributing to the model.

Key concepts

Model Utility Test

The model utility test in multiple regression analysis is critical for understanding the overall significance of the model. It is, essentially, a hypothesis test that checks whether there is a statistically significant relationship between the response variable and the set of predictors.

The null hypothesis (\(H_{0}\)) typically states that none of the predictor variables is significantly related to the output variable—implying that all regression coefficients are equal to zero. On the other hand, the alternative hypothesis (\(H_{a}\)) asserts that at least one of the coefficients is not zero. To perform this test, statisticians usually use the F-statistic, a value derived from an F-distribution that compares the explained variance of the model against the unexplained variance.

If the calculated F-statistic is greater than the critical value from the F-distribution table at a certain significance level (commonly, 0.05), it justifies rejecting the null hypothesis. This means our regression model does provide a better fit to the data than a model with no predictors at all. Correspondingly, if the F-statistic is lower than the critical value, there's no statistical evidence to claim that our model is useful.

R-squared (\rR^2)

The R-squared (\(R^2\)) value is a popular statistic used to gauge the effectiveness of a regression model. It represents the proportion of variance in the dependent variable that can be explained by the independent variables in the model. In other words, it measures the strength of the relationship between the model and the dependent variable on a scale from 0 to 1, where a higher value typically suggests a better model fit.

However, one must be cautious; a higher R-squared does not necessarily indicate that the model is the best. It simply tells us how much of the variability in the dependent variable our model can explain. Still, a low R-squared—as in the exercise where it was reported to be 0.16—could imply a weak relationship between the variables or that key predictors might be missing from the model, leading to questions about the model's predictive power.

Adjusted R-squared

While the R-squared value can give us a quick indication of a model's explanatory power, it has a significant limitation: it can increase simply by adding more predictors, regardless of whether they are meaningful to the model. This is where Adjusted R-squared comes into play.

The Adjusted R-squared adjusts the R-squared value for the number of predictors in the model, penalizing for adding predictors that do not improve the model. This statistic is particularly useful when comparing models with a different number of predictors. If we have a model where the adjusted R-squared is substantially lower than the R-squared, it might indicate that some predictors are not contributing to the model and could be removed.

Comparing R-squared and Adjusted R-squared helps to ensure that our model is not just fitting the data better because we've added more variables, but because the variables we've added truly carry explanatory power.

F-statistic

The F-statistic plays a central role in conducting the model utility test discussed earlier. It is calculated from an ANOVA (analysis of variance) and is used to compare model fits—essentially, whether any of the independent variables, when taken together, are related to the dependent variable.

The formula to calculate it is relatively straightforward: \[ F = \frac{R^{2} / p}{(1-R^{2}) / (n - p - 1)} \] where 'p' is the number of predictors and 'n' is the total sample size. Once the F-statistic is determined, it is compared with a critical value from an F-distribution table. A high F-statistic, which indicates a significant amount of variance explained by the model relative to the amount of unexplained variance, leads to the rejection of the null hypothesis—acknowledging that the regression model provides a better fit than the intercept-only model.

Understanding the F-statistic helps in validating the effectiveness of the model and ensuring that the results of the regression analysis are reliable and not due to random chance.