Question

Question: Find an SVD of each matrix in Exercise 5-12. (Hint: In Exercise 11, one choice for U is \(\left( {\begin{array}{*{20}{c}}{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}\end{array}} \right)\). In Exercise 12, one column of U can be \(\left( {\begin{array}{*{20}{c}}{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\\{ - \frac{{\bf{2}}}{{\sqrt {\bf{6}} }}}\\{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\end{array}} \right)\).)

8. \(\left( {\begin{array}{*{20}{c}}{\bf{4}}&{\bf{6}}\\{\bf{0}}&{\bf{4}}\end{array}} \right)\)

Short answer

The singular value decomposition of \(A\) is, \(A = \left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{ - \frac{1}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}8&0\\0&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\\{ - \frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\)

Step-by-step solution

The Singular Value Decomposition

Consider \(m \times n\) matrix with rank \(r\) as \(A\). There is a \(m \times n\) matrix \(\sum \) as seen in (3) wherein the diagonal entriesin \(D\) are the first \(r\) singular valuesof A, \({\sigma _1} \ge \cdots \ge {\sigma _r} > 0,\) and there arises an \(m \times m\) orthogonal matrix \(U\) and an \(n \times n\) orthogonal matrix \(V\) such that \(A = U\sum {V^T}\).

Find the eigenvalues of the given matrix

Let \(A = \left( {\begin{array}{*{20}{c}}4&6\\0&4\end{array}} \right)\). The transpose of A is:

\({A^T} = \left( {\begin{array}{*{20}{c}}4&0\\6&4\end{array}} \right)\)

Find the product \({A^T}A\).

\(\begin{array}{c}{A^T}A = \left( {\begin{array}{*{20}{c}}4&0\\6&4\end{array}} \right)\left( {\begin{array}{*{20}{c}}4&6\\0&4\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{16}&{24}\\{24}&{52}\end{array}} \right)\end{array}\)

The characteristic equation for \({A^T}A\) is:

\(\begin{array}{c}\det \left| {{A^T}A - \lambda I} \right| = 0\\\left| {\begin{array}{*{20}{c}}{16 - \lambda }&{24}\\{24}&{52 - \lambda }\end{array}} \right| = 0\\\left( {16 - \lambda } \right)\left( {52 - \lambda } \right) - 576 = 0\\{\lambda ^2} - 68\lambda + 256 = 0\\\lambda = 64,4\end{array}\)

Find the eigenvectors of the given matrix

The eigenvectors of the matrix for \(\lambda = 64\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}{ - 48}&{24}\\{24}&{ - 12}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\ - 48{x_1} + 24{x_2} = 0\\24{x_1} - 12{x_2} = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \left( {\begin{array}{*{20}{c}}{\frac{1}{2}}\\1\end{array}} \right)\;{\rm{or}}\,\left( {\begin{array}{*{20}{c}}1\\2\end{array}} \right)\)

The eigenvectors of the matrix for \(\lambda = 4\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}{12}&{24}\\{24}&{48}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\12{x_1} + 24{x_2} = 0\\24{x_1} + 48{x_2} = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \,\left( {\begin{array}{*{20}{c}}{ - 2}\\1\end{array}} \right)\)

Find the Eigenvector matrix

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}1\\2\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_1} = \frac{1}{{\sqrt {{1^2} + {2^2}} }}\left( {\begin{array}{*{20}{c}}1\\2\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}{ - 2}\\1\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_2} = \frac{1}{{\sqrt {{{\left( { - 2} \right)}^2} + {1^2}} }}\left( {\begin{array}{*{20}{c}}{ - 2}\\1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{ - \frac{2}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

The eigenvector matrix is:

\(\begin{array}{c}V = \left( {\begin{array}{*{20}{c}}{{{\bf{v}}_1}}&{{{\bf{v}}_2}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{ - \frac{2}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

Find the value of \(\Sigma \)

The singular value is the square root of eigenvalues of \({A^T}A\), that are 8 and 2. Thus the matrix \(\Sigma \) is:

\(\Sigma = \left( {\begin{array}{*{20}{c}}8&0\\0&2\end{array}} \right)\)

Find the matrix U

Compute the column vectors \({{\bf{u}}_1}\) and \({{\bf{u}}_2}\):

\(\begin{array}{c}{{\bf{u}}_1} = \frac{1}{{\sqrt {{\lambda _1}} }}A{{\bf{v}}_1}\\ = \frac{1}{{\sqrt {64} }}\left( {\begin{array}{*{20}{c}}4&6\\0&4\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

And

\(\begin{array}{c}{{\bf{u}}_2} = \frac{1}{{\sqrt {{\lambda _2}} }}A{{\bf{v}}_2}\\ = \frac{1}{{\sqrt 4 }}\left( {\begin{array}{*{20}{c}}4&6\\0&4\end{array}} \right)\left( {\begin{array}{*{20}{c}}{ - \frac{2}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{ - \frac{1}{{\sqrt 5 }}}\\{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

The matrix Ucan be determined as follows:

\(\begin{array}{c}U = \left( {\begin{array}{*{20}{c}}{{{\bf{u}}_1}}&{{{\bf{u}}_2}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{ - \frac{1}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

The SVD of A can be written as,

\(\begin{array}{c}A = U\Sigma {V^T}\\ = \left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{ - \frac{1}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}8&0\\0&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\\{ - \frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\end{array}\)

Hence, the SVD of the given matrix is, \(A = \left( {\begin{array}{*{20}{c}}{\frac{2}{{\sqrt 5 }}}&{ - \frac{1}{{\sqrt 5 }}}\\{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}8&0\\0&2\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 5 }}}&{\frac{2}{{\sqrt 5 }}}\\{ - \frac{2}{{\sqrt 5 }}}&{\frac{1}{{\sqrt 5 }}}\end{array}} \right)\).