Question

In Exercises 9-12, find (a) the orthogonal projection of b onto \({\bf{Col}}A\) and (b) a least-squares solution of \(A{\bf{x}} = {\bf{b}}\).

10. \(A = \left[ {\begin{aligned}{{}{}}{\bf{1}}&{\bf{2}}\\{ - {\bf{1}}}&{\bf{4}}\\{\bf{1}}&{\bf{2}}\end{aligned}} \right]\), \({\bf{b}} = \left[ {\begin{aligned}{{}{}}{\bf{3}}\\{ - {\bf{1}}}\\{\bf{5}}\end{aligned}} \right]\)

Short answer

1. The orthogonal projection of bonto Col A is \(\left[ {\begin{aligned}{{}{}}4\\{ - 1}\\4\end{aligned}} \right]\).
2. The least-square solution is \(\left[ {\begin{aligned}{{}{}}3\\{\frac{1}{2}}\end{aligned}} \right]\).

Step-by-step solution

Find the orthogonal projection of \({\bf{\hat b}}\)

The orthogonal projection of b onto \({\rm{Col}}A\) is:

\(\begin{aligned}{}{\bf{\hat b}} &= {\rm{pro}}{{\rm{j}}_{{\rm{col}}A}}{\bf{b}}\\ & = \frac{{{\bf{b}} \cdot {{\bf{v}}_1}}}{{{{\bf{v}}_1} \cdot {{\bf{v}}_1}}}{{\bf{v}}_1} + \frac{{{\bf{b}} \cdot {{\bf{v}}_2}}}{{{{\bf{v}}_2} \cdot {{\bf{v}}_2}}}{{\bf{v}}_2}\\ & = \frac{{\left[ {\begin{aligned}{{}{}}3&{ - 1}&5\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}1\\{ - 1}\\1\end{aligned}} \right]}}{{\left[ {\begin{aligned}{{}{}}1&{ - 1}&1\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}1\\{ - 1}\\1\end{aligned}} \right]}}{{\bf{v}}_1} + \frac{{\left[ {\begin{aligned}{{}{}}3&{ - 1}&5\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}2\\4\\2\end{aligned}} \right]}}{{\left[ {\begin{aligned}{{}{}}2&4&2\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}2\\4\\2\end{aligned}} \right]}}{{\bf{v}}_2}\\ & = \frac{9}{3}\left[ {\begin{aligned}{{}{}}1\\{ - 1}\\1\end{aligned}} \right] + \frac{{12}}{{24}}\left[ {\begin{aligned}{{}{}}2\\4\\2\end{aligned}} \right]\end{aligned}\)

Solve further,

\(\begin{aligned}{}{\bf{\hat b}} & = \left[ {\begin{aligned}{{}{}}{3 + 1}\\{ - 3 + 2}\\{3 + 1}\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}4\\{ - 1}\\4\end{aligned}} \right]\end{aligned}\)

The orthogonal projection of b onto ColA is \(\left[ {\begin{aligned}{{}{}}4\\{ - 1}\\4\end{aligned}} \right]\).

Find the normal equation

Find the product \({A^T}A\).

\(\begin{aligned}{}{A^T}A = \left[ {\begin{aligned}{{}{}}1&{ - 1}&1\\2&4&2\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}1&2\\{ - 1}&4\\1&2\end{aligned}} \right]\\ = \left[ {\begin{aligned}{{}{}}3&0\\0&{24}\end{aligned}} \right]\end{aligned}\)

Find the product \({A^T}{\bf{b}}\).

\(\begin{aligned}{}{A^T}{\bf{b}} = \left[ {\begin{aligned}{{}{}}1&{ - 1}&1\\2&4&2\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}3\\{ - 1}\\5\end{aligned}} \right]\\ = \left[ {\begin{aligned}{{}{}}9\\{12}\end{aligned}} \right]\end{aligned}\)

The normal equation can be written as:

\(\begin{aligned}{}\left( {{A^T}A} \right){\bf{x}} = {A^T}{\bf{b}}\\\left[ {\begin{aligned}{{}{}}3&0\\0&{24}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}{{x_1}}\\{{x_2}}\end{aligned}} \right] = \left[ {\begin{aligned}{{}{}}9\\{12}\end{aligned}} \right]\end{aligned}\)

Find the least square solution

The least-square solution can be calculated as follows:

\(\begin{aligned}{}{\bf{\hat x}} & = {\left( {{A^T}A} \right)^{ - 1}}{A^T}{\bf{b}}\\ & = {\left[ {\begin{aligned}{{}{}}3&0\\0&{24}\end{aligned}} \right]^{ - 1}}\left[ {\begin{aligned}{{}{}}9\\{12}\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}{\frac{1}{3}}&0\\0&{\frac{1}{{24}}}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}9\\{12}\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}3\\{\frac{1}{2}}\end{aligned}} \right]\end{aligned}\)

Thus, the least square solution is \(\left[ {\begin{aligned}{{}{}}3\\{\frac{1}{2}}\end{aligned}} \right]\).