Question

Consider a subspace V of ${\mathbf{\square }}^{\mathbf{n}}$with a basis ${\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{1}}\mathbf{,}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{2}}\mathbf{,}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{,}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{m}}$; suppose we wish to find a formula for the orthogonal projection onto V. Using the methods we have developed thus far, we can proceed in two steps: We use the Gram-Schmidt process to construct an orthonormal basis${\stackrel{\mathbf{\rightharpoonup }}{\mathbf{u}}}_{\mathbf{1}}\mathbf{,}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{,}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{u}}}_{\mathbf{m}}$ of V, and then we use Theorem 5.3.10: The matrix of the orthogonal projection QQ^T, where $\mathit{Q}\mathbf{=}\left[{\stackrel{\rightharpoonup }{u}}_1...{\stackrel{\rightharpoonup }{u}}_m\right]$. In this exercise we will see how we write the matrix of the projection directly in terms of the basis${\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{1}}\mathbf{,}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{,}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{m}}$ and the matrix $\mathit{A}\mathbf{=}\left[{\stackrel{\rightharpoonup }{v}}_1...{\stackrel{\rightharpoonup }{v}}_m\right]$. (This issue will be discussed more thoroughly in Section 5.4: see theorem 5.4.7.)

Since $\mathit{p}\mathit{r}\mathit{o}{\mathit{j}}_{\mathbf{v}}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}$ is in V, we can write $\mathit{p}\mathit{r}\mathit{o}{\mathit{j}}_{\mathbf{v}}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}\mathbf{=}\mathit{c}\mathbf{,}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{1}}\mathbf{+}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{+}{\mathit{c}}_{\mathbf{m}}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{m}}$for some scalars ${\mathit{c}}_{\mathbf{1}}\mathbf{,}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{,}{\mathit{c}}_{\mathbf{m}}$yet to be determined. Now $\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}\mathbf{-}\mathit{p}\mathit{r}\mathit{o}{\mathit{j}}_{\mathbf{v}}\left(\stackrel{\rightharpoonup }{x}\right)\mathbf{=}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}\mathbf{-}{\mathit{c}}_{\mathbf{1}}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{1}}\mathbf{-}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{-}{\mathit{c}}_{\mathbf{m}}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{m}}$ is orthogonal to V, meaning that role="math" localid="1660623171035" ${\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{i}}\mathbf{·}\mathbf{(}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}\mathbf{-}{\mathbf{c}}_{\mathbf{1}}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{1}}\mathbf{-}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{-}{\mathbf{c}}_{\mathbf{m}}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{m}}\mathbf{)}\mathbf{=}\mathbf{0}$for $\mathit{i}\mathbf{=}\mathbf{1}\mathbf{,}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{,}\mathit{m}$.

1. Use the equation${\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{i}}\mathbf{·}\mathbf{(}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}\mathbf{-}{\mathbf{c}}_{\mathbf{1}}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{1}}\mathbf{-}\mathbf{.}\mathbf{.}\mathbf{.}\mathbf{-}{\mathbf{c}}_{\mathbf{m}}{\stackrel{\mathbf{\rightharpoonup }}{\mathbf{v}}}_{\mathbf{m}}\mathbf{)}\mathbf{=}\mathbf{0}$ to show that ${\mathit{A}}^{\mathbf{T}}\mathit{A}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{c}}\mathbf{=}{\mathit{A}}^{\mathbf{T}}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}$, where $\stackrel{\mathbf{\rightharpoonup }}{\mathbf{c}}\mathbf{=}\left[\begin{array}{c}{c}_1\\ ⋮\\ c_m\end{array}\right]$.
2. Conclude that$\stackrel{\mathbf{\rightharpoonup }}{\mathbf{c}}\mathbf{=}{\left(A^{T}A\right)}^{\mathbf{-}\mathbf{1}}{\mathit{A}}^{\mathbf{T}}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}$ and $\mathit{p}\mathit{r}\mathit{o}{\mathit{j}}_{\mathbf{v}}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}\mathbf{=}\mathit{A}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{c}}\mathbf{=}\mathit{A}{\left(A^{T}A\right)}^{\mathbf{-}\mathbf{1}}{\mathit{A}}^{\mathbf{T}}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}$.

Short answer

1. It is proved that $A^{T}A\stackrel{\rightharpoonup }{c}=A^T\stackrel{\rightharpoonup }{x}$using the equation ${\stackrel{\rightharpoonup }{v}}_i·\left(x-c_1{\stackrel{\rightharpoonup }{v}}_1-...-c_m{\stackrel{\rightharpoonup }{v}}_m\right)=0$.
2. It is concluded thatrole="math" localid="1660624017029" $\stackrel{\rightharpoonup }{c}={\left(A^{T}A\right)}^{-1}{A}^T\stackrel{\rightharpoonup }{x}$and role="math" localid="1660626446185" $pro{j}_v\stackrel{\rightharpoonup }{x}=A\stackrel{\rightharpoonup }{c}=A{\left(A^{T}A\right)}^{-1}{A}^T\stackrel{\rightharpoonup }{x}$.

Step-by-step solution

Determine Orthogonal matrix

An $\mathit{n}\mathbf{\times }\mathit{n}$matrix $\mathit{A}$is considered orthogonal if and only if ${\mathit{A}}^{\mathbf{T}}\mathit{A}\mathbf{=}{\mathit{l}}_{\mathbf{n}}$or we can say ${\mathit{A}}^{\mathbf{-}\mathbf{1}}\mathbf{=}{\mathit{A}}^{\mathbf{T}}$.

Determine proof of part (a)

a.

Given that $\stackrel{\rightharpoonup }{c}=\left[\begin{array}{c}{c}_1\\ ⋮\\ c_m\end{array}\right]$. Let a matrix A be $A=\left[{\stackrel{\rightharpoonup }{v}}_1...{\stackrel{\rightharpoonup }{v}}_m\right]$. So, the value of $A^{T}A\stackrel{\rightharpoonup }{c}$can be obtained as follows:

$\begin{array}{rcl}{A}^{T}A\stackrel{\rightharpoonup }{c}& =& A^T\left[{\stackrel{\rightharpoonup }{v}}_1...{\stackrel{\rightharpoonup }{v}}_m\right]\left[\begin{array}{c}{c}_1\\ ⋮\\ c_m\end{array}\right]\\ & =& A^T\left[c_1{\stackrel{\rightharpoonup }{v}}_1...c_m{\stackrel{\rightharpoonup }{v}}_m\right]\end{array}$

Now, we have role="math" localid="1660625187075" $\begin{array}{rcl}{\stackrel{\rightharpoonup }{v}}_i·\left(\stackrel{\rightharpoonup }{x}-c_1{\stackrel{\rightharpoonup }{v}}_1-...-c_m{\stackrel{\rightharpoonup }{v}}_m\right)& =& 0\end{array}$. This can be manipulated as:

$\begin{array}{rcl}{\stackrel{\rightharpoonup }{v}}_i·\left(\stackrel{\rightharpoonup }{x}-c_1{\stackrel{\rightharpoonup }{v}}_1-...-c_m{\stackrel{\rightharpoonup }{v}}_m\right)& =& 0\\ \left(\stackrel{\rightharpoonup }{x}-c_1{\stackrel{\rightharpoonup }{v}}_1-...-c_m{\stackrel{\rightharpoonup }{v}}_m\right)& =& 0\\ \stackrel{\rightharpoonup }{x}& =& c_1{\stackrel{\rightharpoonup }{v}}_1+...+c_m{\stackrel{\rightharpoonup }{v}}_m\end{array}$

So, the value of$A^{T}A\stackrel{\rightharpoonup }{c}$ will become:

$\begin{array}{rcl}{A}^{T}A\stackrel{\rightharpoonup }{c}& =& A^T\left[c_1{\stackrel{\rightharpoonup }{v}}_1...c_m{\stackrel{\rightharpoonup }{v}}_m\right]\\ & =& A^T\stackrel{\rightharpoonup }{x}\end{array}$

Thus, it is proved that$A^{T}A\stackrel{\rightharpoonup }{c}=A^T\stackrel{\rightharpoonup }{x}$ using the equation $\begin{array}{rcl}{\stackrel{\rightharpoonup }{v}}_i·\left(\stackrel{\rightharpoonup }{x}-c_1{\stackrel{\rightharpoonup }{v}}_1-...-c_m{\stackrel{\rightharpoonup }{v}}_m\right)& =& 0\end{array}$.

Determine the proof of part (b)

b.

From part (a), we have $A^{T}A\stackrel{\rightharpoonup }{c}=A^T\stackrel{\rightharpoonup }{x}$. On multiplying${\left(A^{T}A\right)}^{-1}$ both sides to the equation, we get role="math" localid="1660625828176" ${\left(A^{T}A\right)}^{-1}\left(A^{T}A\stackrel{\rightharpoonup }{c}\right)={\left(A^{T}A\right)}^{-1}\left(A^T\stackrel{\rightharpoonup }{x}\right)$. Simplify the equation as follows:

$\begin{array}{rcl}{\left(A^{T}A\right)}^{-1}\left(A^{T}A\stackrel{\rightharpoonup }{c}\right)& =& {\left(A^{T}A\right)}^{-1}\left(A^T\stackrel{\rightharpoonup }{x}\right)\\ {\left(A^{T}A\right)}^{-1}\left(A^{T}A\right)\left(\stackrel{\rightharpoonup }{c}\right)& =& {\left(A^{T}A\right)}^{-1}\left(A^T\stackrel{\rightharpoonup }{x}\right)\\ \stackrel{\rightharpoonup }{c}& =& {\left(A^{T}A\right)}^{-1}\left(A^T\stackrel{\rightharpoonup }{x}\right)\end{array}$

It is known that role="math" localid="1660626003168" $pro{j}_v\stackrel{\rightharpoonup }{x}=c_1{\stackrel{\rightharpoonup }{v}}_1+...+c_m{\stackrel{\rightharpoonup }{v}}_m$. It can be manipulated as:

$\begin{array}{rcl}pro{j}_v\stackrel{\rightharpoonup }{x}& =& c_1{\stackrel{\rightharpoonup }{v}}_1+...+c_m{\stackrel{\rightharpoonup }{v}}_m\\ & =& \left[{\stackrel{\rightharpoonup }{v}}_1...{\stackrel{\rightharpoonup }{v}}_m\right]\left[\begin{array}{c}{c}_1\\ ⋮\\ c_m\end{array}\right]\\ & =& A\stackrel{\rightharpoonup }{c}\end{array}$

Now, we have the value ${\left(A^{T}A\right)}^{-1}\left(A^T\stackrel{\rightharpoonup }{x}\right)$. Put this value in the above equation to get:

$\begin{array}{rcl}pro{j}_v\stackrel{\rightharpoonup }{x}& =& A\stackrel{\rightharpoonup }{c}\\ & =& A\left({\left(A^{T}A\right)}^{-1}\left(A^T\stackrel{\rightharpoonup }{x}\right)\right)\\ & =& A{\left(A^{T}A\right)}^{-1}\left(A^T\stackrel{\rightharpoonup }{x}\right)\end{array}$

Thus, it is concluded that$\stackrel{\rightharpoonup }{c}={\left(A^{T}A\right)}^{-1}{A}^T\stackrel{\rightharpoonup }{x}$ and $pro{j}_v\stackrel{\rightharpoonup }{x}=A\stackrel{\rightharpoonup }{c}=A{\left(A^{T}A\right)}^{-1}{A}^T\stackrel{\rightharpoonup }{x}$.