Question

Question: Find an SVD of each matrix in Exercise 5-12. (Hint: In Exercise 11, one choice for U is \(\left( {\begin{array}{*{20}{c}}{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}\end{array}} \right)\). In Exercise 12, one column of U can be \(\left( {\begin{array}{*{20}{c}}{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\\{ - \frac{{\bf{2}}}{{\sqrt {\bf{6}} }}}\\{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\end{array}} \right)\).)

11. \(\left( {\begin{array}{*{20}{c}}{ - {\bf{3}}}&{\bf{1}}\\{\bf{6}}&{ - {\bf{2}}}\\{\bf{6}}&{ - {\bf{2}}}\end{array}} \right)\)

Short answer

The singular value decomposition of \(A\) is, \(A = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{3}}&{\frac{2}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{ - \frac{1}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{\frac{2}{3}}&{ - \frac{1}{3}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{3\sqrt {10} }&0\\0&0\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{3}{{\sqrt {10} }}}&{ - \frac{1}{{\sqrt {10} }}}\\{\frac{1}{{\sqrt {10} }}}&{\frac{3}{{\sqrt {10} }}}\end{array}} \right)\).

Step-by-step solution

The Singular Value Decomposition

Consider \(m \times n\) matrix with rank \(r\) as \(A\). There is a \(m \times n\) matrix \(\sum \) as seen in (3) wherein the diagonal entriesin \(D\) are the first \(r\) singular valuesof A, \({\sigma _1} \ge \cdots \ge {\sigma _r} > 0,\) and there arises an \(m \times m\) orthogonal matrix \(U\) and an \(n \times n\) orthogonal matrix \(V\) such that \(A = U\sum {V^T}\).

Find the eigenvalues of the given matrix

Let \(A = \left( {\begin{array}{*{20}{c}}{ - 3}&1\\6&{ - 2}\\6&{ - 2}\end{array}} \right)\). The transpose of A is:

\({A^T} = \left( {\begin{array}{*{20}{c}}{ - 3}&6&6\\1&{ - 2}&{ - 2}\end{array}} \right)\)

Find the product \({A^T}A\).

\(\begin{array}{c}{A^T}A = \left( {\begin{array}{*{20}{c}}{ - 3}&6&6\\1&{ - 2}&{ - 2}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{ - 3}&1\\6&{ - 2}\\6&{ - 2}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{81}&{ - 27}\\{ - 27}&9\end{array}} \right)\end{array}\)

The characteristic equation for \({A^T}A\) is:

\(\begin{array}{c}\det \left| {{A^T}A - \lambda I} \right| = 0\\\left| {\begin{array}{*{20}{c}}{81 - \lambda }&{ - 27}\\{ - 27}&{9 - \lambda }\end{array}} \right| = 0\\{\lambda ^2} - 90\lambda = 0\\\lambda = 0,90\end{array}\)

Find the eigenvectors of the given matrix

The eigenvectors of the matrix for \(\lambda = 0\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}{81}&{ - 27}\\{ - 27}&9\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\3{x_1} - {x_2} = 0\\ - 27{x_1} + 9{x_2} = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \,\left( {\begin{array}{*{20}{c}}1\\3\end{array}} \right)\)

The eigenvectors of the matrix for \(\lambda = 90\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}{ - 9}&{ - 27}\\{ - 27}&{ - 81}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\ - {x_1} - 3{x_2} = 0\\ - {x_1} - 3{x_2} = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \,\left( {\begin{array}{*{20}{c}}3\\{ - 1}\end{array}} \right)\)

Find the Eigenvector matrix

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}1\\3\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_1} = \frac{1}{{\sqrt {{1^2} + {3^2}} }}\left( {\begin{array}{*{20}{c}}1\\3\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt {10} }}}\\{\frac{3}{{\sqrt {10} }}}\end{array}} \right)\end{array}\)

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}3\\{ - 1}\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_2} = \frac{1}{{\sqrt {{3^2} + {{\left( { - 1} \right)}^2}} }}\left( {\begin{array}{*{20}{c}}3\\{ - 1}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{3}{{\sqrt {10} }}}\\{ - \frac{1}{{\sqrt {10} }}}\end{array}} \right)\end{array}\)

The eigenvector matrix is:

\(\begin{array}{c}V = \left( {\begin{array}{*{20}{c}}{{{\bf{v}}_1}}&{{{\bf{v}}_2}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt {10} }}}&{\frac{3}{{\sqrt {10} }}}\\{\frac{3}{{\sqrt {10} }}}&{ - \frac{1}{{\sqrt {10} }}}\end{array}} \right)\end{array}\)

Find the value of \(\Sigma \)

The singular value is the square root of eigenvalues of \({A^T}A\) i.e. \(3\sqrt {10} \) and 0. Thus the matrix \(\Sigma \) is:

\(\Sigma = \left( {\begin{array}{*{20}{c}}{3\sqrt {10} }&0\\0&0\\0&0\end{array}} \right)\)

Find the matrix U

Find the column vector \({{\bf{u}}_1}\):

\(\begin{array}{c}{{\bf{u}}_1} = \frac{1}{{\sqrt \lambda }}A{V_1}\\ = \frac{1}{{\sqrt {90} }}\left( {\begin{array}{*{20}{c}}{ - 3}&1\\6&{ - 2}\\6&{ - 2}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt {10} }}}\\{ - \frac{3}{{\sqrt {10} }}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{3}}\\{\frac{2}{3}}\\{\frac{2}{3}}\end{array}} \right)\end{array}\)

The vectors \({{\bf{u}}_2}\) and \({{\bf{u}}_3}\) are orthogonal to \({{\bf{u}}_1}\). Therefore,

\(\begin{array}{c}{\bf{u}}_1^T{\bf{x}} = 0\\ - {{\bf{x}}_1} + 2{{\bf{x}}_2} + 2{{\bf{x}}_3} = 0\end{array}\)

Find the column vector \({{\bf{u}}_2}\):

For \({x_1} = 2\), \({x_2} = 2\) and \({x_3} = - 1\).

\({{\bf{u}}_3} = \left( {\begin{array}{*{20}{c}}{\frac{2}{3}}\\{\frac{2}{3}}\\{ - \frac{1}{3}}\end{array}} \right)\)

Find the column vector \({{\bf{u}}_3}\):

For \({x_1} = 2\), \({x_2} = 2\) and \({x_3} = - 1\)

The matrix U can be written as,

\(U = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{3}}&{\frac{2}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{ - \frac{1}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{\frac{2}{3}}&{ - \frac{1}{3}}\end{array}} \right)\)

The SVD of A can be written as,

\(\begin{array}{c}A = U\Sigma {V^T}\\ = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{3}}&{\frac{2}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{ - \frac{1}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{\frac{2}{3}}&{ - \frac{1}{3}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{3\sqrt {10} }&0\\0&0\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{3}{{\sqrt {10} }}}&{ - \frac{1}{{\sqrt {10} }}}\\{\frac{1}{{\sqrt {10} }}}&{\frac{3}{{\sqrt {10} }}}\end{array}} \right)\end{array}\)

Hence, the SVD of the given matrix is, \(A = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{3}}&{\frac{2}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{ - \frac{1}{3}}&{\frac{2}{3}}\\{\frac{2}{3}}&{\frac{2}{3}}&{ - \frac{1}{3}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{3\sqrt {10} }&0\\0&0\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{3}{{\sqrt {10} }}}&{ - \frac{1}{{\sqrt {10} }}}\\{\frac{1}{{\sqrt {10} }}}&{\frac{3}{{\sqrt {10} }}}\end{array}} \right)\).