Question

In Exercises 1-4, find a least-sqaures solution of \(A{\bf{x}} = {\bf{b}}\) by (a) constructing a normal equations for \({\bf{\hat x}}\) and (b) solving for \({\bf{\hat x}}\).

1. \(A = \left[ {\begin{aligned}{{}{}}{ - {\bf{1}}}&{\bf{2}}\\{\bf{2}}&{ - {\bf{3}}}\\{ - {\bf{1}}}&{\bf{3}}\end{aligned}} \right]\), \({\bf{b}} = \left[ {\begin{aligned}{{}{}}{\bf{4}}\\{\bf{1}}\\{\bf{2}}\end{aligned}} \right]\)

Short answer

(a) \(\left[ {\begin{aligned}{{}{}}{ - 4}\\{11}\end{aligned}} \right]\)

(b)\(\left[ {\begin{aligned}{{}{}}3\\2\end{aligned}} \right]\)

Step-by-step solution

(a) Step 1: Find the products \({A^T}A\) and \({A^T}{\bf{b}}\)

Find the product \({A^T}A\).

\(\begin{aligned}{}{A^T}A &= \left[ {\begin{aligned}{{}{}}{ - 1}&2&{ - 1}\\2&{ - 3}&3\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}{ - 1}&2\\2&{ - 3}\\{ - 1}&3\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{{}{}}6&{ - 11}\\{ - 11}&{22}\end{aligned}} \right]\end{aligned}\)

Find the product \({A^T}{\bf{b}}\).

\(\begin{aligned}{}{A^T}{\bf{b}} &= \left[ {\begin{aligned}{{}{}}{ - 1}&2&{ - 1}\\2&{ - 3}&3\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}4\\1\\2\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{{}{}}{ - 4}\\{11}\end{aligned}} \right]\end{aligned}\)

Find the solution by constructing the normal equations

The normal equations can be written as:

\(\begin{aligned}{}\left( {{A^T}A} \right){\bf{x}} = {A^T}{\bf{b}}\\\left[ {\begin{aligned}{{}{}}6&{ - 11}\\{ - 11}&{22}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}{{x_1}}\\{{x_2}}\end{aligned}} \right] = \left[ {\begin{aligned}{{}{}}{ - 4}\\{11}\end{aligned}} \right]\end{aligned}\)

(b) Step 3: Find the component \({\bf{\hat x}}\)

The component \({\bf{\hat x}}\) can be calculated as:

\(\begin{aligned}{}{\bf{\hat x}} &= {\left( {{A^T}A} \right)^{ - 1}}\left( {{A^T}{\bf{b}}} \right)\\ &= {\left[ {\begin{aligned}{{}{}}6&{ - 11}\\{ - 11}&{22}\end{aligned}} \right]^{ - 1}}\left[ {\begin{aligned}{{}{}}{ - 4}\\{11}\end{aligned}} \right]\\ &= \frac{1}{{11}}\left[ {\begin{aligned}{{}{}}{22}&{11}\\{11}&6\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}{ - 4}\\{11}\end{aligned}} \right]\\ &= \frac{1}{{11}}\left[ {\begin{aligned}{{}{}}{33}\\{22}\end{aligned}} \right]\\ &= \left[ {\begin{aligned}{{}{}}3\\2\end{aligned}} \right]\end{aligned}\)

The \({\bf{\hat x}}\) component is \(\left[ {\begin{aligned}{{}{}}3\\2\end{aligned}} \right]\).