Question

In Exercises 1-4, find a least-sqaures solution of \(A{\bf{x}} = {\bf{b}}\) by (a) constructing a normal equations for \({\bf{\hat x}}\) and (b) solving for \({\bf{\hat x}}\).

4. \(A = \left( {\begin{aligned}{{}{}}{\bf{1}}&{\bf{3}}\\{\bf{1}}&{ - {\bf{1}}}\\{\bf{1}}&{\bf{1}}\end{aligned}} \right)\), \({\bf{b}} = \left( {\begin{aligned}{{}{}}{\bf{5}}\\{\bf{1}}\\{\bf{0}}\end{aligned}} \right)\)

Short answer

(a) \(\left( {\begin{aligned}{{}{}}3&3\\3&{11}\end{aligned}} \right)\left( {\begin{aligned}{{}{}}{{x_1}}\\{{x_2}}\end{aligned}} \right) = \left( {\begin{aligned}{{}{}}6\\{14}\end{aligned}} \right)\)

(b)\(\left( {\begin{aligned}{{}{}}1\\1\end{aligned}} \right)\)

Step-by-step solution

(a) Step 1: Find the products \({A^T}A\) and \({A^T}{\bf{b}}\)

Find the product \({A^T}A\).

\(\begin{aligned}{}{A^T}A &= \left( {\begin{aligned}{{}{}}1&1&1\\3&{ - 1}&1\end{aligned}} \right)\left( {\begin{aligned}{{}{}}1&3\\1&{ - 1}\\1&1\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}3&3\\3&{11}\end{aligned}} \right)\end{aligned}\)

Find the product \({A^T}{\bf{b}}\).

\(\begin{aligned}{}{A^T}{\bf{b}} &= \left( {\begin{aligned}{{}{}}1&1&1\\3&{ - 1}&1\end{aligned}} \right)\left( {\begin{aligned}{{}{}}5\\1\\0\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}6\\{14}\end{aligned}} \right)\end{aligned}\)

Find the solution by constructing the normal equations

The normal equations can be written as:

\(\begin{aligned}{}\left( {{A^T}A} \right){\bf{x}} = {A^T}{\bf{b}}\\\left( {\begin{aligned}{{}{}}3&3\\3&{11}\end{aligned}} \right)\left( {\begin{aligned}{{}{}}{{x_1}}\\{{x_2}}\end{aligned}} \right) = \left( {\begin{aligned}{{}{}}6\\{14}\end{aligned}} \right)\end{aligned}\)

(b) Step 3: Find the component \({\bf{\hat x}}\)

The component \({\bf{\hat x}}\) can be calculated as:

\(\begin{aligned}{}{\bf{\hat x}} &= {\left( {{A^T}A} \right)^{ - 1}}\left( {{A^T}{\bf{b}}} \right)\\ &= {\left( {\begin{aligned}{{}{}}3&3\\3&{11}\end{aligned}} \right)^{ - 1}}\left( {\begin{aligned}{{}{}}6\\{14}\end{aligned}} \right)\\ &= \frac{1}{{24}}\left( {\begin{aligned}{{}{}}{24}\\{24}\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}1\\1\end{aligned}} \right)\end{aligned}\)

The \({\bf{\hat x}}\) component is \(\left( {\begin{aligned}{{}{}}1\\1\end{aligned}} \right)\).