Question

Let \(\mathbf{A}=\left[a_{i j}\right]\) be a real symmetric matrix. Prove that \(\sum_{i} \sum_{j} a_{i j}^{2}\) is equal to the sum of the squares of the eigenvalues of \(\mathbf{A}\). Hint: If \(\boldsymbol{\Gamma}\) is an orthogonal matrix, show that \(\sum_{j} \sum_{i} a_{i j}^{2}=\operatorname{tr}\left(\mathbf{A}^{2}\right)=\operatorname{tr}\left(\boldsymbol{\Gamma}^{\prime} \mathbf{A}^{2} \boldsymbol{\Gamma}\right)=\) \(\operatorname{tr}\left[\left(\mathbf{\Gamma}^{\prime} \mathbf{A} \mathbf{\Gamma}\right)\left(\mathbf{\Gamma}^{\prime} \mathbf{A} \boldsymbol{\Gamma}\right)\right]\)

Short answer

The sum of the squares of all elements in a real symmetric matrix is equal to the sum of the squares of the eigenvalues of that matrix. This relies on the properties of symmetric matrices, orthogonal transformations and eigenvalues.

Step-by-step solution

Understanding Matrix Notation

First, let's note that \(a_{ij}\) represents the value in the i-th row and j-th column of the matrix \(A\). The expression \(\sum_{i} \sum_{j} a_{ij}^2\) simply signifies the sum of the squares of all the entries in the matrix \(A\). The trace of a matrix, noted as tr, is defined as the sum of the diagonal entries in the matrix. This would simplify the problem with the double sums.

Applying Hint

Next, apply the hint given that for an orthogonal matrix \(\Gamma\), we can rewrite the problem as \(\operatorname{tr}\left(\mathbf{A}^2\right) = \operatorname{tr}\left(\boldsymbol{\Gamma}^{\prime} \mathbf{A}^2 \boldsymbol{\Gamma}\right)\). The transpose of \(\Gamma\) is \(\Gamma^{\prime}\), and the product of \(\Gamma^{\prime}\) and \(\Gamma\) is the identity matrix, denoted \(I\). Since \(\mathbf{A}\) is a symmetric matrix, the sum of the squares of its entries (i.e., \(\operatorname{tr}\left(\mathbf{A}^2\right)\)) is equal to the trace of its square.

Making use of the Orthogonality and Apply Hint Again

Using the property that the transpose of the product of two matrices is the product of their transposes in reverse order and the property of the trace that it is invariant under cyclic permutations, the rightmost term can be arranged as \(\operatorname{tr}\left[\left(\mathbf{\Gamma}^{\prime} \mathbf{A} \mathbf{\Gamma}\right)\left(\mathbf{\Gamma}^{\prime} \mathbf{A} \boldsymbol{\Gamma}\right)\right]\) which can be interpreted as the sum of the squares of all entries of the matrix \(\mathbf{\Gamma}^{\prime} \mathbf{A} \mathbf{\Gamma}\).

Eigenvalues of a Symmetric Matrix

Next, make use of the property that eigenvalues of a symmetric matrix are real and the eigenvectors corresponding to different eigenvalues are orthogonal. Because \(\mathbf{A}\) is symmetric, there exists an orthogonal transformation \(\Gamma\) such that \(\Gamma' A \Gamma\) is a diagonal matrix. The elements of the diagonal matrix are the eigenvalues of the original symmetric matrix \(\mathbf{A}\). Hence, the sum of the squares of all entries of that new matrix is really just the sum of the squares of the eigenvalues of \(A\). Therefore, we can conclude that the sum of the squares of all elements in a real symmetric matrix is equal to the sum of the squares of the eigenvalues of the same matrix.

Key concepts

Symmetric Matrix

Understanding what a symmetric matrix is forms the basis for grasping other related concepts. A matrix is considered symmetric if it is equal to its transpose, which is the matrix obtained by flipping it over its diagonal. So mathematically, a matrix \( \mathbf{A} \) is symmetric if \( \mathbf{A} = \mathbf{A}^{\prime} \).

This property implies that the entries across the main diagonal are mirrored. For example, the element at row \(i\) and column \(j\) is the same as the element at row \(j\) and column \(i\). Symmetric matrices are significant because they have real eigenvalues and orthogonal eigenvectors, which have profound implications in various branches of mathematics and physics.

In the context of the exercise, the given matrix \( \mathbf{A} \) is symmetric, and we are exploring a relationship between the sum of squares of its elements and the sum of squares of its eigenvalues.

Orthogonal Matrix

Now let's discuss an orthogonal matrix. A matrix \(\boldsymbol{\Gamma}\) is orthogonal if its transpose is also its inverse, meaning \( \boldsymbol{\Gamma}^{\prime} \boldsymbol{\Gamma} = \mathbf{I} \) where \( \mathbf{I} \) is the identity matrix. Essentially, multiplying an orthogonal matrix by its transpose yields an identity matrix, which has 1s on the diagonal and 0s elsewhere.

Orthogonal matrices often emerge when considering rotations or reflections in a space. They are invaluable in calculations since they preserve lengths and angles, which means they keep the geometry of the space unchanged. In the exercise, orthogonal matrices are employed to simplify the expression for the sum of squares of the matrix's elements, ultimately revealing the relationship with the eigenvalues.

Matrix Trace

The matrix trace, denoted as \( \operatorname{tr}(\mathbf{A}) \), is a relatively simple yet powerful concept. It is defined as the sum of the diagonal elements of a square matrix \( \mathbf{A} \). Formally, \( \operatorname{tr}(\mathbf{A}) = a_{11} + a_{22} + ... + a_{nn} \). The trace has properties that are useful in this context: it is linear and invariant under cyclic permutations.

When we deal with the trace of a product of matrices, such as \( \mathbf{A}^2 \), the trace is equal to the sum of the eigenvalues for that matrix. This ties the trace to the eigenvalues directly and highlights its utility when we're looking to simplify expressions involving the sum of squaress of matrix elements, as seen in the given exercise.

Eigenvalues and Eigenvectors

The concepts of eigenvalues and eigenvectors are central to understanding the behavior of matrices. If \( \mathbf{A} \) is a matrix, an eigenvalue \( \lambda \) and its associated eigenvector \( \mathbf{x} \) satisfy \( \mathbf{A} \mathbf{x} = \lambda \mathbf{x} \). In other words, multiplying the matrix by its eigenvector gives a vector that is merely scaled by the eigenvalue.

For symmetric matrices, eigenvalues are always real, and eigenvectors corresponding to different eigenvalues are orthogonal to one another. This beautiful property of symmetric matrices is exploited in the step-by-step solution of the exercise, where the matrix is diagonalized using its eigenvectors which form an orthogonal matrix. The diagonal elements of the resulting matrix are the eigenvalues, allowing us to directly calculate the sum of the squares of the eigenvalues as required for the proof.