Question

Question: Find an SVD of each matrix in Exercise 5-12. (Hint: In Exercise 11, one choice for U is \(\left( {\begin{array}{*{20}{c}}{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}\end{array}} \right)\). In Exercise 12, one column of U can be \(\left( {\begin{array}{*{20}{c}}{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\\{ - \frac{{\bf{2}}}{{\sqrt {\bf{6}} }}}\\{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\end{array}} \right)\).)

7. \(\left( {\begin{array}{*{20}{c}}{\bf{2}}&{ - {\bf{1}}}\\{\bf{2}}&{\bf{2}}\end{array}} \right)\)

Short answer

The singular value decomposition of \(A\) is, \(A = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{ - \frac{2}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}3&0\\0&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\\{ - \frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\).

Step-by-step solution

The Singular Value Decomposition

Consider \(m \times n\) matrix with rank \(r\) as \(A\). There is a \(m \times n\) matrix \(\sum \) as seen in (3) wherein the diagonal entriesin \(D\) are the first \(r\) singular valuesof A, \({\sigma _1} \ge \cdots \ge {\sigma _r} > 0,\) and there arises an \(m \times m\) orthogonal matrix \(U\) and an \(n \times n\) orthogonal matrix \(V\) such that \(A = U\sum {V^T}\).

Find the eigenvalues of the given matrix

Let \(A = \left( {\begin{array}{*{20}{c}}2&{ - 1}\\2&2\end{array}} \right)\). The transpose of A is:

\({A^T} = \left( {\begin{array}{*{20}{c}}2&2\\{ - 1}&2\end{array}} \right)\)

Find the product \({A^T}A\).

\(\begin{array}{c}{A^T}A = \left( {\begin{array}{*{20}{c}}2&2\\{ - 1}&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}2&{ - 1}\\2&2\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}8&2\\2&5\end{array}} \right)\end{array}\)

The characteristic equation for \({A^T}A\) is:

\(\begin{array}{c}\det \left| {{A^T}A - \lambda I} \right| = 0\\\left| {\begin{array}{*{20}{c}}{8 - \lambda }&2\\2&{5 - \lambda }\end{array}} \right| = 0\\\left( {8 - \lambda } \right)\left( {5 - \lambda } \right) - 4 = 0\\{\lambda ^2} - 13\lambda + 36 = 0\\\lambda = 9,4\end{array}\)

Find the eigenvectors of the given matrix

The eigenvectors of the matrix for \(\lambda = 9\) are:

\({A^T}A - 9I = \left( {\begin{array}{*{20}{c}}{ - 1}&2\\2&{ - 4}\end{array}} \right)\)

Write the row-reduced Augmented matrix as:

\(\begin{array}{c}M = \left( {\begin{array}{*{20}{c}}{ - 1}&2&0\\2&{ - 4}&0\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{ - 1}&2&0\\0&0&0\end{array}} \right)\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\left( {{R_2} \to {R_2} + 2{R_1}} \right)\end{array}\)

So, the corresponding eigenvector is \(\left( {\begin{array}{*{20}{c}}2\\1\end{array}} \right)\).

The eigenvectors of the matrix for \(\lambda = 4\) are:

\({A^T}A - 4I = \left( {\begin{array}{*{20}{c}}4&2\\2&1\end{array}} \right)\)

Write the row-reduced Augmented matrix is:

\(\begin{array}{c}M = \left( {\begin{array}{*{20}{c}}4&2&0\\2&1&0\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}4&2&0\\0&0&0\end{array}} \right)\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\left( {{R_2} \to {R_2} - 2{R_1}} \right)\end{array}\)

So, the corresponding eigenvector is \(\left( {\begin{array}{*{20}{c}}{ - \frac{1}{2}}\\1\end{array}} \right)\).

Find the Eigenvector matrix

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}2\\1\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_1} = \frac{1}{{\sqrt {{2^2} + {1^2}} }}\left( {\begin{array}{*{20}{c}}2\\1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}{ - \frac{1}{2}}\\1\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_2} = \frac{1}{{\sqrt {{{\left( { - \frac{1}{2}} \right)}^2} + {1^2}} }}\left( {\begin{array}{*{20}{c}}{ - \frac{1}{2}}\\1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

The eigenvector matrix is:

\(\begin{array}{c}V = \left( {\begin{array}{*{20}{c}}{{{\bf{v}}_1}}&{{{\bf{v}}_2}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{ - \frac{1}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

Find the value of \(\Sigma \)

The singular value is square root of eigenvalues of \({A^T}A\) i.e. 3 and 2. Thus the matrix \(\Sigma \) is:

\(\Sigma = \left( {\begin{array}{*{20}{c}}3&0\\0&2\end{array}} \right)\)

Find the matrix U

Compute the column vectors \({{\bf{u}}_1}\) and \({{\bf{u}}_2}\):

\(\begin{array}{c}{{\bf{u}}_1} = \frac{1}{{\sqrt {{\lambda _1}} }}A{{\bf{v}}_1}\\ = \frac{1}{{\sqrt 9 }}\left( {\begin{array}{*{20}{c}}2&{ - 1}\\2&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

And

\(\begin{array}{c}{{\bf{u}}_2} = \frac{1}{{\sqrt {{\lambda _2}} }}A{{\bf{v}}_2}\\ = \frac{1}{{\sqrt 4 }}\left( {\begin{array}{*{20}{c}}2&{ - 1}\\2&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{ - \frac{2}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

The matrix Ucan be determined as follows:

\(\begin{array}{c}U = \left( {\begin{array}{*{20}{c}}{{{\bf{u}}_1}}&{{{\bf{u}}_2}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{ - \frac{2}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

The SVD of A can be written as,

\(\begin{array}{c}A = U\Sigma {V^T}\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{ - \frac{2}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}3&0\\0&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\\{ - \frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

Hence, the SVD of the given matrix is \(A = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{ - \frac{2}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}3&0\\0&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\\{ - \frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\).