Question

In Exercises 1-4, find the equation \(y = {\beta _0} + {\beta _1}x\) of the least-square line that best fits the given data points.

1. \(\left( {1,0} \right),\left( {2,1} \right),\left( {4,2} \right),\left( {5,3} \right)\)

Short answer

The equation of the least-square line that best fits is \(y = - 0.6 + 0.7x\)

Step-by-step solution

The design matrix X and observation vector y

Use the x and y coordinates to find the \(X\) and \(y\) matrices.

\(X = \left[ {\begin{aligned}1&1\\1&2\\1&4\\1&5\end{aligned}} \right]\) and \(y = \left[ {\begin{aligned}0\\1\\2\\3\end{aligned}} \right]\)

Obtain the normal equations 

The normal equation of \(X\beta = y\) can be obtained using \({X^T}X\beta = {X^T}y\) which is equivalent to \(\beta = {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\).

Find \({X^T}X\) as follows:

\(\begin{aligned}{X^T}X &= \left[ {\begin{aligned}1&1&1&1\\1&2&4&5\end{aligned}} \right]\left[ {\begin{aligned}1&1\\1&2\\1&4\\1&5\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{1 + 1 + 1 + 1}&{1 + 2 + 4 + 5}\\{1 + 2 + 4 + 5}&{1 + 4 + 16 + 25}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}4&{12}\\{12}&{46}\end{aligned}} \right]\end{aligned}\)

Find the inverse of \({X^T}X\) as follows:

\(\begin{aligned}{\left( {{X^T}X} \right)^{ - 1}} &= {\left[ {\begin{aligned}4&{12}\\{12}&{46}\end{aligned}} \right]^{ - 1}}\\ &= \frac{1}{{184 - 144}}\left[ {\begin{aligned}{46}&{ - 12}\\{ - 12}&4\end{aligned}} \right]\\ &= \frac{1}{{40}}\left[ {\begin{aligned}{46}&{ - 12}\\{ - 12}&4\end{aligned}} \right]\end{aligned}\)

Find \({X^T}y\) as follows:

\(\begin{aligned}{X^T}y &= \left[ {\begin{aligned}1&1&1&1\\1&2&4&5\end{aligned}} \right]\left[ {\begin{aligned}0\\1\\2\\3\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{0 + 1 + 2 + 3}\\{0 + 2 + 8 + 15}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}6\\{25}\end{aligned}} \right]\end{aligned}\)

Solve the normal equation

Substitute the calculated values in \(\beta = {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\) and solve it as follows:

\(\begin{aligned}\beta &= {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\\\beta &= \frac{1}{{40}}\left[ {\begin{aligned}{46}&{ - 12}\\{ - 12}&4\end{aligned}} \right]\left[ {\begin{aligned}6\\{25}\end{aligned}} \right]\\\beta &= \frac{1}{{40}}\left[ {\begin{aligned}{276 - 300}\\{ - 72 + 100}\end{aligned}} \right]\\\beta &= \frac{1}{{40}}\left[ {\begin{aligned}{ - 24}\\{28}\end{aligned}} \right]\\\left[ {\begin{aligned}{{\beta _0}}\\{{\beta _1}}\end{aligned}} \right] &= \left[ {\begin{aligned}{ - 0.6}\\{0.7}\end{aligned}} \right]\end{aligned}\)

Hence, the equation of the least-square line that best fits is \(y = - 0.6 + 0.7x\).