Question

Find the SVD of $$\left(\begin{array}{cc}0& -1\\ 1& 1\\ -1& 0\end{array}\right)$$ showing that the singular values are $\sqrt{3}$ and 1 .

Short answer

The SVD of the matrix is: $$U=\left(\begin{array}{c}\dots \dots \dots \end{array}\right),\mathrm{\Sigma }=\left(\begin{array}{ccc}\sqrt{3}& 00& 1\end{array}\right),V=\left(\begin{array}{c}\dots \dots \dots \end{array}\right)$$

Step-by-step solution

– Calculate A Transpose times A

Let's find the matrix product of the given matrix A and its transpose, A^T. First, the matrix A is: $$A=\left(\begin{array}{cccc}0& -11& 1-1& 0\end{array}\right)$$ The transpose, A^T, is: $$A^T=\left(\begin{array}{ccccc}0& 1& -1-1& 1& 0\end{array}\right)$$ Now, let's multiply A^T by A: $$A^{T}A=\left(\begin{array}{ccccc}0& 1& -1-1& 1& 0\end{array}\right)\left(\begin{array}{cccc}0& -11& 1-1& 0\end{array}\right)=\left(\begin{array}{ccc}2& 11& 2\end{array}\right)$$

– Find the Eigenvalues of A^T A

To find the singular values, first find the eigenvalues of the matrix product from Step 1. The characteristic equation is found by solving: $$\text{det}(A^{T}A-\lambda I)=0$$ where I is the identity matrix and \lambda represents the eigenvalues. Thus, $$\text{det}\left(\begin{array}{ccc}2-\lambda & 11& 2-\lambda \end{array}\right)=0$$ Calculate the determinant: $$(2-\lambda )(2-\lambda )-1\times 1={\lambda }^2-4\lambda +3=0$$ Solving for \lambda, using the quadratic formula, \lambda = \frac{-b \pm \sqrt{b^2-4ac}}{2a} \, we get: $$\lambda =\frac{4\pm \sqrt{(4{)}^2-4\cdot 1\cdot 3}}{2}=\frac{4\pm \sqrt{16-12}}{2}=\frac{4\pm 2}{2}=3,1$$

– Compute the Singular Values

The singular values \sigma are the square roots of the eigenvalues: $${\sigma }_1=\sqrt{3},{\sigma }_2=1$$

– Calculate the Singular Vectors

For each singular value, find the corresponding singular vectors by solving (A - \lambda I) x = 0. For \lambda_1 = 3: $$(A^{T}A-3I)\left(\begin{array}{c}{x}_1{x}_2\end{array}\right)=0$$ This simplifies to: $$\left(\begin{array}{ccc}-1& 11& -1\end{array}\right)\left(\begin{array}{c}{x}_1{x}_2\end{array}\right)=0\Rightarrow x_1=x_2$$ Choose x_1 = \frac{1}{\sqrt{2}}, x_2 = \frac{1}{\sqrt{2}}: $$u_1=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\frac{1}{\sqrt{2}}\end{array}\right)$$ Similarly, for \lambda_2 = 1: $$(A^{T}A-I)\left(\begin{array}{c}{x}_1{x}_2\end{array}\right)=0\Rightarrow \left(\begin{array}{ccc}1& 11& 1\end{array}\right)\left(\begin{array}{c}{x}_1{x}_2\end{array}\right)=0\Rightarrow x_1=-x_2$$ Choose x_1 = \frac{1}{\sqrt{2}}, x_2 = \frac{-1}{\sqrt{2}}: $$u_2=\left(\begin{array}{c}\frac{1}{\sqrt{2}}-\frac{1}{\sqrt{2}}\end{array}\right)$$

– Construct U, Σ, V Matrices

The U matrix is constructed from the unit vectors of A and is orthogonal. The Σ matrix contains the singular values, and the V matrix is the orthogonal eigenvectors from the previous step. The singular values are: $$\mathrm{\Sigma }=\left(\begin{array}{ccc}\sqrt{3}& 00& 1\end{array}\right)$$ The U matrix is: \[ U = \begin{pmatrix} -\frac{1}{\sqrt{3}} & -\frac{1}{\sqrt{6}} \ \frac{2}{\sqrt{3}} & -\frac{1}{\sqrt{6}} \ \frac{0}{\sqrt{3}} & -\frac{\frac{2}{\sqrt{6}} \end{pmatrix} \] The final V matrix is: $$V=\left(\begin{array}{ccc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right)$$

Key concepts

Eigenvalues

Understanding eigenvalues is crucial when learning about Singular Value Decomposition (SVD). At its core, an eigenvalue is a scalar that indicates how much the corresponding eigenvector is stretched during a linear transformation. To find the eigenvalues of a matrix, you solve the characteristic equation:
det(A - λI) = 0, where A is your original matrix, λ (Lambda) represents the eigenvalues, and I is the identity matrix.
By solving this equation, you obtain scalar values that provide insights into the properties of the matrix, like its invertibility and stability.

Eigenvectors

Eigenvectors are vectors that maintain their direction during a linear transformation, although they may get scaled by their associated eigenvalues. To find the eigenvector corresponding to an eigenvalue, you solve the equation
(A - λI)x = 0, where x is the eigenvector.
Eigenvectors are fundamental in breaking down complex transformations into simpler, more manageable parts. They help in many applications, like facial recognition and stability analysis of systems. In the SVD process, eigenvectors of the matrix provide the basis for the singular vectors. Understanding how to calculate and use them enriches your grasp of matrix algebra.

Orthogonal Matrices

Orthogonal matrices play a vital role in SVD. A matrix is orthogonal if its rows and columns are perpendicular unit vectors. Mathematically, a matrix Q is orthogonal if:
Q^T Q = I, where Q^T is the transpose of Q, and I is the identity matrix.
Orthogonality ensures numerical stability and simplifies matrix operations. In SVD, the matrices U and V you derive are orthogonal, facilitating the decomposition process. This orthogonality property means transforming the original matrix doesn’t distort its fundamental geometric relationships.

Matrix Algebra

Matrix algebra provides the foundation for understanding SVD and other linear transformations. Involving operations like addition, subtraction, multiplication, and inversion of matrices, it allows you to manipulate data structures effectively. Key concepts in matrix algebra include:

• Matrix Multiplication
• Determinants
• Inverses
• Eigenvalues and Eigenvectors

Matrices are used to represent linear transformations, and understanding their algebraic properties helps in applications like computer graphics, systems of linear equations, and machine learning. Mastering these operations assists you in tackling complex mathematical problems efficiently.