Question

The data points given in Exercises \(6-7\) were formed by reversing the slope of the lines in Exercises 4 - 5. Plot the points on graph paper and calculater and \(r^{2}\). Notice the change in the sign of \(r\) and the relationship between the values of \(r^{2}\) compared to Exercises \(4-5 .\) By what percentage was the sum of squares of deviations reduced by using the least-squares predictor \(\hat{y}=a+b x\) rather than \(\bar{y}\) as a predictor of \(y\) ? $$\begin{array}{l|llllll}x & 1 & 2 & 3 & 4 & 5 & 6 \\\\\hline y & 0 & 2 & 3 & 5 & 5 & 7\end{array}$$

Short answer

Answer: The sum of squares of deviations was reduced by approximately 73.27% when using the least-squares predictor 𝑦̂ =𝑎+𝑏𝑥 rather than 𝑦¯ as a predictor of 𝑦.

Step-by-step solution

Calculate the mean of x and y values

To determine the mean, we will add up all the values for x and y and divide by the number of values. We have the following data points: $$\begin{array}{l|llllll}x & 1 & 2 & 3 & 4 & 5 & 6 \\\\\hline y & 0 & 2 & 3 & 5 & 5 & 7\end{array}$$ We have 6 data points, so \(\bar{x}=\frac{1+2+3+4+5+6}{6}=\frac{21}{6}=3.5\) and \(\bar{y}=\frac{0+2+3+5+5+7}{6}=\frac{22}{6}=3.\overline{6}\).

Calculate the covariance of x and y and the variance of x

Covariance quantifies the relationship between the x and y values. We will use the following formula to calculate the covariance: $$Cov(x,y)=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{n}$$ Variance of x is calculated as: $$Var(x)=\frac{\sum_{i=1}^{n}(x_i-\bar{x})^2}{n}$$ Cov(x,y)= \(\frac{(1-3.5)(0-3.\overline{6})+\dots+(6-3.5)(7-3.\overline{6})}{6}\approx2.\overline{3}\) Var(x) = \(\frac{(1-3.5)^2+\dots+(6-3.5)^2}{6}\approx2.916\overline{6}\)

Determine the values of a and b

Using the formulas for the regression coefficients a and b, we can now calculate the values of a and b: $$b=\frac{Cov(x,y)}{Var(x)}$$ $$a=\bar{y}-b\bar{x}$$ Using the covariance and variance calculated in step 2, we have $$b\approx \frac{2.\overline{3}}{2.916\overline{6}} \approx 0.916\overline{6}$$ $$a\approx 3.\overline{6} - 0.916\overline{6} * 3.5 \approx -0.\overline{1}$$

Calculate the least-squares predictor for each x value

Now we can calculate the least-squares predictor \(\hat{y}=a+bx\) for each x value: $$\hat{y}\approx -0.\overline{1}+0.916\overline{6}x$$ Using this equation, we obtain the predicted y values for the given x values: $$\begin{array}{l|llllll}x & 1 & 2 & 3 & 4 & 5 & 6 \\\\\hline \hat{y} & 0.\overline{8} & 1.74\overline{2} & 2.66\overline{4} & 3.582 & 4.49\overline{8} & 5.41\overline{04}\end{array}$$

Calculate the sum of squares of deviations using the least-squares predictor and \(\bar{y}\) as a predictor

We will now compute the sum of squares of deviations using the least-squares predictor and \(\bar{y}\) as a predictor: $$SSD(\hat{y})=\sum_{i=1}^{n}(y_i-\hat{y}_i)^2$$ $$SSD(\bar{y})=\sum_{i=1}^{n}(y_i-\bar{y})^2$$ Using the observed and predicted y values, we calculate the sum of squares of deviations for both predictors: $$SSD(\hat{y})\approx(0-0.\overline{8})^2+(2-1.74\overline{2})^2+\dots+(7-5.41\overline{04})^2\approx5.87\overline{75}$$ $$SSD(\bar{y})\approx(0-3.\overline{6})^2+(2-3.\overline{6})^2+\dots+(7-3.\overline{6})^2\approx22$$

Determine the percentage reduction in the sum of squares of deviations

Finally, we can calculate the percentage reduction in the sum of squares of deviations by using the least-squares predictor \(\hat{y}=a+bx\) rather than \(\bar{y}\) as a predictor of \(y\): $$\text{Percentage Reduction}=\frac{SSD(\bar{y})-SSD(\hat{y})}{SSD(\bar{y})}\times 100$$ Using the calculated values from step 5, we obtain: $$\text{Percentage Reduction}\approx\frac{22-5.87\overline{75}}{22}\times100\approx73.\overline{27}\%$$ The sum of squares of deviations was reduced by approximately \(73.\overline{27}\%\) when using the least-squares predictor \(\hat{y}=a+bx\) rather than \(\bar{y}\) as a predictor of \(y\).

Key concepts

Sum of Squares of Deviations

The sum of squares of deviations (SSD) is a measure that captures the total deviation of data points from their expected values. Essentially, it indicates how much the observed data spread around a given line, such as the least-squares regression line. In mathematical terms, if we have observed values of a variable, say \(y_i\), and a set of predicted values \(\hat{y}_i\), the SSD is calculated as:
\[ SSD = \sum_{i=1}^{n}(y_i - \hat{y}_i)^2 \]
This formula squares the differences between the predicted and actual values, with the squaring process giving more weight to larger deviations. A lower SSD indicates that the model has a better fit to the data since the observations are closer to the line. The exercise's solution compared SSD using the least-squares predictor against using the mean of \(y\) as a predictor, demonstrating a significant reduction in the SSD and, thereby, showing the effectiveness of the regression model.

Correlation Coefficient (r)

The correlation coefficient, represented by \(r\), is a statistical metric expressing the degree of linear relationship between two variables. It ranges between -1 and +1, where +1 indicates a perfect positive linear correlation, -1 indicates a perfect negative linear correlation, and 0 signifies no linear correlation. A change in the sign of \(r\) implies a change in the direction of the relationship between the variables. In our context, calculating \(r\) helps in understanding how well the regression line represents the relationship in the data. The square of the correlation coefficient, \(r^2\), represents the proportion of variability in one variable that can be explained by its linear relationship with the other variable. In the given exercise, the relationship between the change in \(r\) and \(r^2\) compared to earlier exercises provides insights into the nature and strength of the linear association between the variables.

Variance and Covariance

Variance and covariance are concepts that describe how data points vary around the mean. Variance is a measure of the spread of a set of numbers. If we denote the mean of the observations as \(\bar{x}\), the variance of these observations \(Var(x)\) is calculated as:
\[ Var(x) = \frac{\sum_{i=1}^{n}(x_i - \bar{x})^2}{n} \]
Covariance, on the other hand, extends this idea to two variables, showing how both variables vary with each other. For two sets of numbers \(x\) and \(y\), with their means being \(\bar{x}\) and \(\bar{y}\), respectively, covariance \(Cov(x,y)\) is calculated using the formula:
\[ Cov(x,y) = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{n} \]
These two metrics together are the backbone of linear regression analysis as they are used to calculate the regression line, capturing the relationship between the dependent variable \(y\) and independent variable \(x\).

Linear Regression Analysis

Linear regression analysis is used to model the relationship between a dependent variable and one or more independent variables. It assumes a linear relationship between the variables, characterized by the equation of the line \(\hat{y} = a + bx\), where \(a\) is the intercept, \(b\) is the slope, and \(\hat{y}\) are the predicted values. The goal is to find the line that best fits the data by minimizing the sum of squares of deviations. In the provided exercise, we looked at each step needed to construct this model from calculating the mean values, variance, and covariance, through to determining the regression coefficients and finally, the best-fit line. The resulting model's effectiveness was illustrated by a significant percentage reduction in the SSD when comparing predictions using the least-squares regression line versus predictions using the mean value of \(y\). The essence of linear regression is therefore to provide the most accurate predictions of the dependent variable, given the values of independent variables, harnessing the power of both variance and covariance.