Question

a. Rewrite the data in Example 1 with new \(x\)-coordinates in mean deviation form. Let \(X\) be the associated design matrix. Why are the columns of \(X\) orthogonal?

b. Write the normal equations for the data in part (a), and solve them to find the least-squares line, \(y = {\beta _0} + {\beta _1}x*\), where \(x* = x - 5.5\).

Short answer

(a) The data in mean deviation form can be written as:

\(X = \left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)\)

As the sum of all entries of the second column is 0, so the columns of are orthogonal.

(b) The normal equation is \(\left( {\begin{aligned}4&0\\0&{21}\end{aligned}} \right)\left( {\begin{aligned}{{\beta _1}}\\{{\beta _2}}\end{aligned}} \right) = \left( {\begin{aligned}9\\{7.5}\end{aligned}} \right)\). The least-squares line is \(y = \frac{9}{4} + \frac{5}{{14}}\left( {x - 5.5} \right)\).

Step-by-step solution

Find the mean of the given data

(a) The data in example 1 is: \(\left( {2,1} \right),\left( {5,2} \right),\left( {7,3} \right)\) and \(\left( {8,3} \right)\).

Find the mean of the data.

\(\begin{aligned}\bar x &= \frac{{\sum x }}{n}\\ &= \frac{{2 + 5 + 7 + 8}}{4}\\ &= \frac{{22}}{4}\\ &= 5.5\end{aligned}\)

So, the mean of the data is \(5.5\).

Find the value of \(x*\)

It is given that, \(x* = x - 5.5\). Find \(x* = x - 5.5\) for different values of \(x\) in the given data set.

\(\begin{aligned}x* = 2 - 5.5\\ = - 3.5\end{aligned}\)

\(\begin{aligned}x* = 5 - 5.5\\ = - 0.5\end{aligned}\)

\(\begin{aligned}x* = 7 - 5.5\\ = 1.5\end{aligned}\)

\(\begin{aligned}x* = 8 - 5.5\\ = 2.5\end{aligned}\)

So, the data in mean deviation form can be written as,

\(X = \left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)\)

Determine whether \(X\) is orthogonal

On adding the elements of the second column of \(X\), the value is 0, so the columns of \(X\) are orthogonal.

The General Linear Model

The equation of the general linear model is given as,

\({\bf{y}} = X\beta + \in \)

Here, \({\bf{y}} = \left( {\begin{aligned}{{y_1}}\\{{y_2}}\\ \vdots \\{{y_n}}\end{aligned}} \right)\) is an observational vector, \(X = \left( {\begin{aligned}1&{{x_1}}& \cdots &{x_1^n}\\1&{{x_2}}& \cdots &{x_2^n}\\ \vdots & \vdots & \ddots & \vdots \\1&{{x_n}}& \cdots &{x_n^n}\end{aligned}} \right)\) is the design matrix, \(\beta = \left( {\begin{aligned}{{\beta _1}}\\{{\beta _2}}\\ \vdots \\{{\beta _n}}\end{aligned}} \right)\) is parameter vector, and \( \in = \left( {\begin{aligned}{{ \in _1}}\\{{ \in _2}}\\ \vdots \\{{ \in _n}}\end{aligned}} \right)\) is a residual vector.

Find design matrix, observation vector, parameter vector for given data 

Write the design matrix and observational vector for the given data points.

Design matrix: \(X = \left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)\)

Observational matrix: \({\bf{y}} = \left( {\begin{aligned}1\\2\\3\\3\end{aligned}} \right)\)

And the parameter vectorfor the given equation is,

\({\bf{\beta }} = \left( {\begin{aligned}{{\beta _0}}\\{{\beta _1}}\end{aligned}} \right)\)

Normal equation

The normal equation is given by,

\({X^T}X\beta = {X^T}{\bf{y}}\)

Find least-squares line

(b)

The normal equation is\({X^T}X\beta = {X^T}{\bf{y}}\), find \(\beta \) for the given data by using the normal equation.

\(\begin{aligned}\left( {{{\left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)}^T}\left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)} \right)\beta & = {\left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)^T}\left( {\begin{aligned}1\\2\\3\\3\end{aligned}} \right)\\\left( {\begin{aligned}4&0\\0&{21}\end{aligned}} \right)\beta & = \left( {\begin{aligned}9\\{7.5}\end{aligned}} \right)\\\left( {\begin{aligned}{{\beta _1}}\\{{\beta _2}}\end{aligned}} \right) &= {\left( {\begin{aligned}4&0\\0&{21}\end{aligned}} \right)^{ - 1}}\left( {\begin{aligned}9\\{7.5}\end{aligned}} \right)\\ &= \frac{1}{{84}}\left( {\begin{aligned}{21}&0\\0&4\end{aligned}} \right)\left( {\begin{aligned}9\\{7.5}\end{aligned}} \right)\end{aligned}\)

Solve further,

\(\begin{aligned}\left( {{{\left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)}^T}\left( {\begin{aligned}1&{ - 3.5}\\1&{ - 0.5}\\1&{1.5}\\1&{2.5}\end{aligned}} \right)} \right)\beta & = \left( {\begin{aligned}{\frac{1}{4}}&0\\0&{\frac{1}{{21}}}\end{aligned}} \right)\left( {\begin{aligned}9\\{7.5}\end{aligned}} \right)\\ &= \left( {\begin{aligned}{\frac{9}{4}}\\{\frac{{7.5}}{{21}}}\end{aligned}} \right)\\ &= \left( {\begin{aligned}{\frac{9}{4}}\\{\frac{5}{{14}}}\end{aligned}} \right)\end{aligned}\)

Substitute the obtained values into \(y = {\beta _0} + {\beta _1}x*\).

\(y = \frac{9}{4} + \frac{5}{{14}}\left( {x - 5.5} \right)\)

Hence, the required least-squares line is \(y = \frac{9}{4} + \frac{5}{{14}}\left( {x - 5.5} \right)\).