Question

If \(A\) is \(n \times n\), show that every eigenvalue of \(A^{T} A\) is nonnegative. [Hint: Compute \(\|A \mathbf{x}\|^{2}\) where \(\mathbf{x}\) is an eigenvector.]

Short answer

The eigenvalues of \(A^{T}A\) are nonnegative because \(\|A \mathbf{x}\|^2 \geq 0\) for any eigenvector \(\mathbf{x}\).

Step-by-step solution

Understanding the Condition

Given that \( A \) is an \( n \times n \) matrix, we need to show that every eigenvalue of \( A^T A \) is nonnegative.

Consider Eigenvalues and Eigenvectors

Let \( \mathbf{v} \) be an eigenvector of \( A^T A \) with corresponding eigenvalue \( \lambda \). This means \( A^T A \mathbf{v} = \lambda \mathbf{v} \).

Compute the Norm

Consider the squared norm \( \|A \mathbf{v}\|^2 = (A\mathbf{v})^T (A\mathbf{v}) \).

Expression via Dot Product

Re-express the squared norm, \( (A\mathbf{v})^T (A\mathbf{v}) = \mathbf{v}^T A^T A \mathbf{v} \).

Substitute the Eigenvalue Expression

Substitute the expression from Step 2: \( \mathbf{v}^T A^T A \mathbf{v} = \mathbf{v}^T (\lambda \mathbf{v}) = \lambda \|\mathbf{v}\|^2 \).

Evaluate the Norm

Since squared norms are always non-negative, \( \|A \mathbf{v}\|^2 \geq 0 \).

Conclude the Eigenvalue Non-negativity

Therefore, \( \lambda \|\mathbf{v}\|^2 \geq 0 \). Since \( \|\mathbf{v}\|^2 > 0 \), it follows that \( \lambda \geq 0 \). Consequently, every eigenvalue of \( A^T A \) is nonnegative.

Key concepts

Eigenvectors

In the study of linear algebra, an eigenvector is a vital concept. An eigenvector of a matrix is a non-zero vector that, when multiplied by the matrix, results in a vector that is a scalar multiple of the original. In simpler terms, if you have a matrix \( A \) and it acts on a vector \( \mathbf{v} \), the result is just the vector \( \mathbf{v} \) stretched by some factor, which we call the eigenvalue. For example, for a matrix \( A \), if \( A \mathbf{v} = \lambda \mathbf{v} \), then \( \mathbf{v} \) is the eigenvector and \( \lambda \) is the eigenvalue.
Breaking it down:

• Eigenvectors are direction vectors that are merely scaled by their corresponding eigenvalues when transformed by their matrix.
• They remain in the same direction throughout the transformation.

Eigenvectors help simplify many matrix operations, particularly useful in fields like machine learning, physics, and other areas involving linear transformations.

Nonnegative Eigenvalues

The concept of nonnegative eigenvalues arises when dealing with specific types of matrices. For example, if \( A \) is an \( n \times n \) matrix, then the product \( A^T A \) (where \( A^T \) is the transpose of \( A \)) is symmetric and positive semi-definite. Positive semi-definite matrices have eigenvalues that are nonnegative.
Here's why:

• Given an eigenvector \( \mathbf{v} \) of \( A^T A \) with eigenvalue \( \lambda \), consider the expression: \( \|A \mathbf{v}\|^2 = \mathbf{v}^T A^T A \mathbf{v} \).
• This expression simplifies to \( \lambda \|\mathbf{v}\|^2 \), where \( \|\mathbf{v}\|^2 \) is always positive for a non-zero eigenvector.
• Thus, the eigenvalue \( \lambda \) must be nonnegative since the squared norm is nonnegative.

This property ensures stability and often appears in optimization problems, confirming that solutions are feasible and practical.

Matrix Transpose

The transpose of a matrix is a simple yet significant concept. When you take the transpose of a matrix \( A \), denoted as \( A^T \), you flip the matrix over its diagonal. This exchanging of rows and columns can turn row vectors into column vectors and vice versa.
Key aspects of transpose:

• It retains the original matrix's dimensions but inverts them, such that the first row becomes the first column.
• For symmetric matrices, where \( A = A^T \), the matrix remains unchanged even after transposing.
• The operation of transposing matrices is handy in many algebraic operations, preserving the inner product and affecting the product of matrices.

Matrix transposition is integral when computing products such as \( A^T A \), which often emerge in methods like data fitting or multivariate statistics.

Norm Computation

The norm of a vector provides a measure of its magnitude. In essence, it gives us the "length" of the vector. There are different types of norms, but the most commonly used is the Euclidean norm, often denoted as \( \|\mathbf{x}\| \). For a vector \( \mathbf{x} \) composed of elements \( x_1, x_2, ..., x_n \), the Euclidean norm is expressed as:\[ \|\mathbf{x}\| = \sqrt{x_1^2 + x_2^2 + ... + x_n^2} \]Important points on norm computation:

• Norms are always non-negative, as they measure the size or magnitude of vectors, disregarding direction.
• They are essential in suitability tests during operations, ensuring boundaries and consistency.
• While used predominantly in vector calculations, they are also crucial in determining matrix properties like conditioning and stability.

Understanding and computing norms is crucial for handling errors and ensuring numerical stability in computational algorithms.