Question

In Exercises 1-4, find the equation \(y = {\beta _0} + {\beta _1}x\) of the least-square line that best fits the given data points.

1. \(\left( {0,1} \right),\left( {1,1} \right),\left( {2,2} \right),\left( {3,2} \right)\)

Short answer

The equation of the least-square line that best fits is \(y = 0.9 + 0.4x\).

Step-by-step solution

The design matrix X and observation vector y

Use the x and y coordinates to find the \(X\) and \(y\) matrices.

\(X = \left[ {\begin{aligned}1&0\\1&1\\1&2\\1&3\end{aligned}} \right]\) and \(y = \left[ {\begin{aligned}1\\1\\2\\2\end{aligned}} \right]\)

Obtain the normal equations 

The normal equation of \(X\beta = y\) can be obtained using \({X^T}X\beta = {X^T}y\) which is equivalent to \(\beta = {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\).

Find \({X^T}X\) as follows:

\(\begin{aligned}{X^T}X &= \left[ {\begin{aligned}1&1&1&1\\0&1&2&3\end{aligned}} \right]\left[ {\begin{aligned}1&0\\1&1\\1&2\\1&3\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{1 + 1 + 1 + 1}&{0 + 1 + 2 + 3}\\{0 + 1 + 2 + 3}&{0 + 1 + 4 + 9}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}4&6\\6&{14}\end{aligned}} \right]\end{aligned}\)

Find the inverse of \({X^T}X\) as follows:

\(\begin{aligned}{\left( {{X^T}X} \right)^{ - 1}} &= {\left[ {\begin{aligned}4&6\\6&{14}\end{aligned}} \right]^{ - 1}}\\ &= \frac{1}{{56 - 36}}\left[ {\begin{aligned}{*{20}{c}}{14}&{ - 6}\\{ - 6}&4\end{aligned}} \right]\\ &= \frac{1}{{20}}\left[ {\begin{aligned}{14}&{ - 6}\\{ - 6}&4\end{aligned}} \right]\end{aligned}\)

Find \({X^T}y\) as follows:

\(\begin{aligned}{X^T}y &= \left[ {\begin{aligned}1&1&1&1\\0&1&2&3\end{aligned}} \right]\left[ {\begin{aligned}1\\1\\2\\2\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{1 + 1 + 2 + 2}\\{0 + 1 + 4 + 6}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}6\\{11}\end{aligned}} \right]\end{aligned}\)

Solve the normal equation

Substitute the calculated values in \(\beta = {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\) and solve it as follows:

\(\begin{aligned}\beta &= {\left( {{X^T}X} \right)^{ - 1}}{X^T}y\\\beta &= \frac{1}{{20}}\left[ {\begin{aligned}{14}&{ - 6}\\{ - 6}&4\end{aligned}} \right]\left[ {\begin{aligned}6\\{11}\end{aligned}} \right]\\\beta &= \frac{1}{{20}}\left[ {\begin{aligned}{84 - 66}\\{ - 36 + 44}\end{aligned}} \right]\\\beta &= \frac{1}{{20}}\left[ {\begin{aligned}{18}\\8\end{aligned}} \right]\\\left[ {\begin{aligned}{{\beta _0}}\\{{\beta _1}}\end{aligned}} \right] &= \left[ {\begin{aligned}{0.9}\\{0.4}\end{aligned}} \right]\end{aligned}\)

Hence, the equation of the least-square line that best fits is \(y = 0.9 + 0.4x\).