Question

In Exercises 1-4, find the equation \(y = {\beta _0} + {\beta _1}x\) of the least-square line that best fits the given data points.

1. \(\left( { - 1,0} \right),\left( {0,1} \right),\left( {1,2} \right),\left( {2,4} \right)\)

Short answer

The equation of the least-square line that best fits is \(y = 1.1 + 1.3x\).

Step-by-step solution

The design matrix X and observation vector y

Use the x and y coordinates to find the \(X\) and \(y\) matrices.

\(X = \left[ {\begin{aligned}1&{ - 1}\\1&0\\1&1\\1&2\end{aligned}} \right]\) and \(y = \left[ {\begin{aligned}0\\1\\2\\4\end{aligned}} \right]\)

Obtain the normal equations

The normal equation of \(X\beta = y\) can be obtained using \({X^T}X\beta = {X^T}y\) which is equivalent to \(\beta = {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\).

Find \({X^T}X\) as follows:

\(\begin{aligned}{X^T}X &= \left[ {\begin{aligned}1&1&1&1\\{ - 1}&0&1&2\end{aligned}} \right]\left[ {\begin{aligned}1&{ - 1}\\1&0\\1&1\\1&2\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{1 + 1 + 1 + 1}&{ - 1 + 0 + 1 + 2}\\{ - 1 + 0 + 1 + 2}&{1 + 0 + 1 + 4}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}4&2\\2&6\end{aligned}} \right]\end{aligned}\)

Find the inverse of \({X^T}X\) as follows:

\(\begin{aligned}{\left( {{X^T}X} \right)^{ - 1}} &= {\left[ {\begin{aligned}4&2\\2&6\end{aligned}} \right]^{ - 1}}\\ &= \frac{1}{{24 - 4}}\left[ {\begin{aligned}6&{ - 2}\\{ - 2}&4\end{aligned}} \right]\\ &= \frac{1}{{20}}\left[ {\begin{aligned}6&{ - 2}\\{ - 2}&4\end{aligned}} \right]\end{aligned}\)

Find \({X^T}y\) as follows:

\(\begin{aligned}{X^T}y &= \left[ {\begin{aligned}1&1&1&1\\{ - 1}&0&1&2\end{aligned}} \right]\left[ {\begin{aligned}0\\1\\2\\4\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{0 + 1 + 2 + 4}\\{0 + 0 + 2 + 8}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}7\\{10}\end{aligned}} \right]\end{aligned}\)

Solve the normal equation

Substitute the calculated values in \(\beta = {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\) and solve it as follows:

\(\begin{aligned}\beta &= {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\\\beta &= \frac{1}{{20}}\left[ {\begin{aligned}6&{ - 2}\\{ - 2}&4\end{aligned}} \right]\left[ {\begin{aligned}7\\{10}\end{aligned}} \right]\\\beta &= \frac{1}{{20}}\left[ {\begin{aligned}{42 - 20}\\{ - 14 + 40}\end{aligned}} \right]\\\beta &= \frac{1}{{20}}\left[ {\begin{aligned}{22}\\{26}\end{aligned}} \right]\\\left[ {\begin{aligned}{{\beta _0}}\\{{\beta _1}}\end{aligned}} \right] &= \left[ {\begin{aligned}{1.1}\\{1.3}\end{aligned}} \right]\end{aligned}\)

Hence, the equation of the least-square line that best fits is \(y = 1.1 + 1.3x\).