Question

Question: Find an SVD of each matrix in Exercise 5-12. (Hint: In Exercise 11, one choice for U is \(\left( {\begin{array}{*{20}{c}}{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}\end{array}} \right)\). In Exercise 12, one column of U can be \(\left( {\begin{array}{*{20}{c}}{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\\{ - \frac{{\bf{2}}}{{\sqrt {\bf{6}} }}}\\{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\end{array}} \right)\).)

10. \(\left( {\begin{array}{*{20}{c}}{\bf{3}}&{ - {\bf{3}}}\\{\bf{0}}&{\bf{0}}\\{\bf{1}}&{\bf{1}}\end{array}} \right)\)

Short answer

The singular value decomposition of \(A\) is, \(A = \left( {\begin{array}{*{20}{c}}{ - 1}&0&1\\0&0&1\\0&1&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}{3\sqrt 2 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\\{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\end{array}} \right)\).

Step-by-step solution

The Singular Value Decomposition

Consider \(m \times n\) matrix with rank \(r\) as \(A\). There is a \(m \times n\) matrix \(\sum \) as seen in (3) wherein the diagonal entries in \(D\) are the first \(r\) singular valuesof A, \({\sigma _1} \ge \cdots \ge {\sigma _r} > 0,\) and there arises an \(m \times m\) orthogonal matrix \(U\) and an \(n \times n\) orthogonal matrix \(V\) such that \(A = U\sum {V^T}\).

Find the eigenvalues of the given matrix

Let \(A = \left( {\begin{array}{*{20}{c}}3&{ - 3}\\0&0\\1&1\end{array}} \right)\). The transpose of A is:

\({A^T} = \left( {\begin{array}{*{20}{c}}3&0&1\\{ - 3}&0&1\end{array}} \right)\)

Find the product \({A^T}A\).

\(\begin{array}{c}{A^T}A = \left( {\begin{array}{*{20}{c}}3&0&1\\{ - 3}&0&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}3&{ - 3}\\0&0\\1&1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{10}&{ - 8}\\{ - 8}&{10}\end{array}} \right)\end{array}\)

The characteristic equation for \({A^T}A\) is:

\(\begin{array}{c}\det \left| {{A^T}A - \lambda I} \right| = 0\\\left| {\begin{array}{*{20}{c}}{10 - \lambda }&{ - 8}\\{ - 8}&{10 - \lambda }\end{array}} \right| = 0\\{\left( {10 - \lambda } \right)^2} - 64 = 0\\\lambda = 18,\,\,2\end{array}\)

Find the eigenvectors of the given matrix

The eigenvectors of the matrix for \(\lambda = 18\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}{ - 8}&{ - 8}\\{ - 8}&{ - 8}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \,\left( {\begin{array}{*{20}{c}}{ - 1}\\1\end{array}} \right)\)

The eigenvectors of the matrix for \(\lambda = 10\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}8&{ - 8}\\{ - 8}&8\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \,\left( {\begin{array}{*{20}{c}}1\\1\end{array}} \right)\)

Find the Eigenvector matrix

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}{ - 1}\\1\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_1} = \frac{1}{{\sqrt {{{\left( { - 1} \right)}^2} + {1^2}} }}\left( {\begin{array}{*{20}{c}}{ - 1}\\1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{{ - 1}}{{\sqrt 2 }}}\\{\frac{1}{{\sqrt 2 }}}\end{array}} \right)\end{array}\)

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}1\\1\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_2} = \frac{1}{{\sqrt {{1^2} + {1^2}} }}\left( {\begin{array}{*{20}{c}}1\\1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 2 }}}\\{\frac{1}{{\sqrt 2 }}}\end{array}} \right)\end{array}\)

The eigenvector matrix is:

\(\begin{array}{c}V = \left( {\begin{array}{*{20}{c}}{{{\bf{v}}_1}}&{{{\bf{v}}_2}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\\{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\end{array}} \right)\end{array}\)

Find the value of \(\Sigma \)

The singular value is the square root of eigenvalues of \({A^T}A\) i.e. \(3\sqrt 2 \) and \(\sqrt 2 \). Thus, the matrix \(\Sigma \) is:

\(\Sigma = \left( {\begin{array}{*{20}{c}}{3\sqrt 2 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\)

Find the matrix U

Let \(U = \left( {\begin{array}{*{20}{c}}a&b&c\\d&e&f\\g&h&i\end{array}} \right)\). Consider the following equation:

\(\begin{array}{c}U\Sigma = AV\\\left( {\begin{array}{*{20}{c}}a&b&c\\d&e&f\\g&h&i\end{array}} \right)\left( {\begin{array}{*{20}{c}}{3\sqrt 2 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right) = \left( {\begin{array}{*{20}{c}}3&{ - 3}\\0&0\\1&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\\{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\end{array}} \right)\\\left( {\begin{array}{*{20}{c}}{3\sqrt 2 a}&{\sqrt 2 b}\\{3\sqrt 2 d}&{\sqrt 2 e}\\{3\sqrt 2 g}&{\sqrt 2 h}\end{array}} \right) = \left( {\begin{array}{*{20}{c}}{ - \frac{6}{{\sqrt 2 }}}&0\\0&0\\0&{\frac{2}{{\sqrt 2 }}}\end{array}} \right)\end{array}\)

So, the matrix \(U = \left( {\begin{array}{*{20}{c}}{ - 1}&0&1\\0&0&1\\0&1&1\end{array}} \right)\).

The SVD of A can be written as,

\(\begin{array}{c}A = U\Sigma {V^T}\\ = \left( {\begin{array}{*{20}{c}}{ - 1}&0&1\\0&0&1\\0&1&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}{3\sqrt 2 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\\{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\end{array}} \right)\end{array}\)

Hence, the SVD of the given matrix is, \(A = \left( {\begin{array}{*{20}{c}}{ - 1}&0&1\\0&0&1\\0&1&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}{3\sqrt 2 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\\{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 2 }}}\end{array}} \right)\).