Question

Question: Find an SVD of each matrix in Exercise 5-12. (Hint: In Exercise 11, one choice for U is \(\left( {\begin{array}{*{20}{c}}{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}\\{\frac{{\bf{2}}}{{\bf{3}}}}&{\frac{{\bf{2}}}{{\bf{3}}}}&{ - \frac{{\bf{1}}}{{\bf{3}}}}\end{array}} \right)\). In Exercise 12, one column of U can be \(\left( {\begin{array}{*{20}{c}}{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\\{ - \frac{{\bf{2}}}{{\sqrt {\bf{6}} }}}\\{\frac{{\bf{1}}}{{\sqrt {\bf{6}} }}}\end{array}} \right)\).

12. \(\left( {\begin{array}{*{20}{c}}{\bf{1}}&{\bf{1}}\\{\bf{0}}&{\bf{1}}\\{ - {\bf{1}}}&{\bf{1}}\end{array}} \right)\)

Short answer

The singular value decomposition of \(A\) is, \(A = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 3 }}}&{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\\{\frac{1}{{\sqrt 3 }}}&0&{ - 2\sqrt 6 }\\{\frac{1}{{\sqrt 3 }}}&{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\sqrt 3 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}0&1\\1&0\end{array}} \right)\).

Step-by-step solution

The Singular Value Decomposition

Consider \(m \times n\) matrix with rank \(r\) as \(A\). There is a \(m \times n\) matrix \(\sum \) as seen in (3) wherein the diagonal entriesin \(D\) are the first \(r\) singular valuesof A, \({\sigma _1} \ge \cdots \ge {\sigma _r} > 0,\) and there arises an \(m \times m\) orthogonal matrix \(U\) and an \(n \times n\) orthogonal matrix \(V\) such that \(A = U\sum {V^T}\).

Find the eigenvalues of the given matrix

Let \(A = \left( {\begin{array}{*{20}{c}}1&1\\0&1\\{ - 1}&1\end{array}} \right)\). The transpose of A is:

\({A^T} = \left( {\begin{array}{*{20}{c}}1&0&{ - 1}\\1&1&1\end{array}} \right)\)

Find the product \({A^T}A\).

\(\begin{array}{c}{A^T}A = \left( {\begin{array}{*{20}{c}}1&0&{ - 1}\\1&1&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}1&1\\0&1\\{ - 1}&1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}2&0\\0&3\end{array}} \right)\end{array}\)

The characteristic equation for \({A^T}A\) is:

\(\begin{array}{c}\det \left| {{A^T}A - \lambda I} \right| = 0\\\left| {\begin{array}{*{20}{c}}{2 - \lambda }&0\\0&{3 - \lambda }\end{array}} \right| = 0\\\lambda = 2,3\end{array}\)

Find the eigenvectors of given matrix

The eigenvectors of the matrix for \(\lambda = 2\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}0&0\\0&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\0{x_1} + {x_2} = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \,\left( {\begin{array}{*{20}{c}}0\\1\end{array}} \right)\)

The eigenvectors of the matrix for \(\lambda = 3\) are:

\(\begin{array}{c}\left( {{A^T}A - \lambda I} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\\left( {\begin{array}{*{20}{c}}{ - 1}&0\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = 0\\ - {x_1} + 0{x_2} = 0\end{array}\)

The general solution is:

\(\left( {\begin{array}{*{20}{c}}{{x_1}}\\{{x_2}}\end{array}} \right) = \,\left( {\begin{array}{*{20}{c}}1\\0\end{array}} \right)\)

Find the Eigenvector matrix

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}0\\1\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_1} = \frac{1}{{\sqrt {{0^2} + {1^2}} }}\left( {\begin{array}{*{20}{c}}0\\1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}0\\1\end{array}} \right)\end{array}\)

Normalize the eigenvector \(\left( {\begin{array}{*{20}{c}}1\\0\end{array}} \right)\) is:

\(\begin{array}{c}{{\bf{v}}_2} = \frac{1}{{\sqrt {{1^2} + {0^2}} }}\left( {\begin{array}{*{20}{c}}1\\0\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}1\\0\end{array}} \right)\end{array}\)

The eigenvector matrix is:

\(\begin{array}{c}V = \left( {\begin{array}{*{20}{c}}{{{\bf{v}}_1}}&{{{\bf{v}}_2}}\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}0&1\\1&0\end{array}} \right)\end{array}\)

Find the value of \(\Sigma \)

The singular value is the square root of eigenvalues of \({A^T}A\) i.e. \(\sqrt 3 \) and \(\sqrt 2 \). Thus, the matrix \(\Sigma \) is:

\(\Sigma = \left( {\begin{array}{*{20}{c}}{\sqrt 3 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\)

Find the matrix U

Find the column vector \({{\bf{u}}_1}\):

\(\begin{array}{c}{{\bf{u}}_1} = \frac{1}{{\sqrt {{\lambda _1}} }}A{{\bf{v}}_1}\\ = \frac{1}{{\sqrt 3 }}\left( {\begin{array}{*{20}{c}}1&1\\0&1\\{ - 1}&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}0\\1\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 3 }}}\\{\frac{1}{{\sqrt 3 }}}\\{\frac{1}{{\sqrt 3 }}}\end{array}} \right)\end{array}\)

And

\(\begin{array}{c}{{\bf{u}}_2} = \frac{1}{{\sqrt {{\lambda _2}} }}A{{\bf{v}}_2}\\ = \frac{1}{{\sqrt 2 }}\left( {\begin{array}{*{20}{c}}1&1\\0&1\\{ - 1}&1\end{array}} \right)\left( {\begin{array}{*{20}{c}}1\\0\end{array}} \right)\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 2 }}}\\0\\{ - \frac{1}{{\sqrt 2 }}}\end{array}} \right)\end{array}\)

The vector \({{\bf{u}}_3}\) is orthogonal to \({{\bf{u}}_1}\) and \({{\bf{u}}_2}\). Therefore,

\(\begin{array}{c}{\bf{u}}_1^T{\bf{x}} = 0\\{x_1} + {x_2} + {x_3} = 0\end{array}\)

And

\(\begin{array}{c}{\bf{u}}_2^T{\bf{x}} = 0\\{x_1} + 0{x_2} - {x_3} = 0\end{array}\)

Then,

\({{\bf{u}}_3} = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 6 }}}\\{ - \frac{2}{{\sqrt 6 }}}\\{\frac{1}{{\sqrt 6 }}}\end{array}} \right)\)

The matrix U can be written as,

\(U = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 3 }}}&{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\\{\frac{1}{{\sqrt 3 }}}&0&{ - 2\sqrt 6 }\\{\frac{1}{{\sqrt 3 }}}&{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\end{array}} \right)\)

The SVD of A can be written as,

\(\begin{array}{c}A = U\Sigma {V^T}\\ = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 3 }}}&{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\\{\frac{1}{{\sqrt 3 }}}&0&{ - 2\sqrt 6 }\\{\frac{1}{{\sqrt 3 }}}&{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\sqrt 3 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}0&1\\1&0\end{array}} \right)\end{array}\)

Hence, the SVD of the given matrix is, \(A = \left( {\begin{array}{*{20}{c}}{\frac{1}{{\sqrt 3 }}}&{\frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\\{\frac{1}{{\sqrt 3 }}}&0&{ - 2\sqrt 6 }\\{\frac{1}{{\sqrt 3 }}}&{ - \frac{1}{{\sqrt 2 }}}&{\frac{1}{{\sqrt 6 }}}\end{array}} \right)\left( {\begin{array}{*{20}{c}}{\sqrt 3 }&0\\0&{\sqrt 2 }\\0&0\end{array}} \right)\left( {\begin{array}{*{20}{c}}0&1\\1&0\end{array}} \right)\).