Question

In Exercises 9-12, find (a) the orthogonal projection of b onto \({\bf{Col}}A\) and (b) a least-squares solution of \(A{\bf{x}} = {\bf{b}}\).

9. \(A = \left[ {\begin{aligned}{{}{}}{\bf{1}}&{\bf{5}}\\{\bf{3}}&{\bf{1}}\\{ - {\bf{2}}}&{\bf{4}}\end{aligned}} \right]\), \({\bf{b}} = \left[ {\begin{aligned}{{}{}}{\bf{4}}\\{ - {\bf{2}}}\\{ - {\bf{3}}}\end{aligned}} \right]\)

Short answer

1. The orthogonal projection of bonto Col A is \(\left[ {\begin{aligned}{{}{}}1\\1\\0\end{aligned}} \right]\).
2. The least-square solution is \(\left[ {\begin{aligned}{{}{}}{\frac{2}{7}}\\{\frac{1}{7}}\end{aligned}} \right]\).

Step-by-step solution

Find the orthogonal projection of \({\bf{\hat b}}\)

The orthogonal projection of b onto \({\rm{Col}}A\) is:

\(\begin{aligned}{}{\bf{\hat b}} &= {\rm{pro}}{{\rm{j}}_{{\rm{col}}A}}{\bf{b}}\\& = \frac{{{\bf{b}} \cdot {{\bf{v}}_1}}}{{{{\bf{v}}_1} \cdot {{\bf{v}}_1}}}{{\bf{v}}_1} + \frac{{{\bf{b}} \cdot {{\bf{v}}_2}}}{{{{\bf{v}}_2} \cdot {{\bf{v}}_2}}}{{\bf{v}}_2}\\ & = \frac{{\left[ {\begin{aligned}{{}{}}4&{ - 2}&{ - 3}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}1\\3\\{ - 2}\end{aligned}} \right]}}{{\left[ {\begin{aligned}{{}{}}1&3&{ - 2}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}1\\3\\{ - 2}\end{aligned}} \right]}}{{\bf{v}}_1} + \frac{{\left[ {\begin{aligned}{{}{}}4&{ - 2}&{ - 3}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}5\\1\\4\end{aligned}} \right]}}{{\left[ {\begin{aligned}{{}{}}5&1&4\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}5\\1\\4\end{aligned}} \right]}}{{\bf{v}}_2}\\ & = \frac{4}{{14}}\left[ {\begin{aligned}{{}{}}1\\3\\{ - 2}\end{aligned}} \right] + \frac{6}{{42}}\left[ {\begin{aligned}{{}{}}5\\1\\4\end{aligned}} \right]\end{aligned}\)

Solve further,

\(\begin{aligned}{}{\bf{\hat b}} & = \left[ {\begin{aligned}{{}{}}{\frac{2}{7} + \frac{5}{7}}\\{\frac{6}{7} + \frac{1}{7}}\\{ - \frac{4}{7} + \frac{4}{7}}\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}1\\1\\0\end{aligned}} \right]\end{aligned}\)

The orthogonal projection of b onto ColA is \(\left[ {\begin{aligned}{{}{}}1\\1\\0\end{aligned}} \right]\).

Find the normal equation

Find the product \({A^T}A\).

\(\begin{aligned}{}{A^T}A & = \left[ {\begin{aligned}{{}{}}1&3&{ - 2}\\5&1&4\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}1&5\\3&1\\{ - 2}&4\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}{14}&0\\0&{42}\end{aligned}} \right]\end{aligned}\)

Find the product \({A^T}{\bf{b}}\).

\(\begin{aligned}{}{A^T}{\bf{b}} & = \left[ {\begin{aligned}{{}{}}1&3&{ - 2}\\5&1&4\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}4\\{ - 2}\\{ - 3}\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}4\\6\end{aligned}} \right]\end{aligned}\)

The normal equation can be written as:

\(\begin{aligned}{}\left( {{A^T}A} \right){\bf{x}} & = {A^T}{\bf{b}}\\\left[ {\begin{aligned}{{}{}}{14}&0\\0&{42}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}{{x_1}}\\{{x_2}}\end{aligned}} \right] & = \left[ {\begin{aligned}{{}{}}4\\6\end{aligned}} \right]\end{aligned}\)

Find the least square solution

The least-square solution can be calculated as follows:

\(\begin{aligned}{}{\bf{\hat x}} & = {\left( {{A^T}A} \right)^{ - 1}}{A^T}{\bf{b}}\\& = {\left[ {\begin{aligned}{{}{}}{14}&0\\0&{42}\end{aligned}} \right]^{ - 1}}\left[ {\begin{aligned}{{}{}}4\\6\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}{\frac{1}{{14}}}&0\\0&{\frac{1}{{42}}}\end{aligned}} \right]\left[ {\begin{aligned}{{}{}}4\\6\end{aligned}} \right]\\ & = \left[ {\begin{aligned}{{}{}}{\frac{2}{7}}\\{\frac{1}{7}}\end{aligned}} \right]\end{aligned}\)

Thus, the least square solution is \(\left[ {\begin{aligned}{{}{}}{\frac{2}{7}}\\{\frac{1}{7}}\end{aligned}} \right]\).