Question

In each case, use the Gram-Schmidt algorithm to find an orthogonal basis of the subspace \(U\), and find the vector in \(U\) closest to \(\mathbf{x}\). a. \(U=\operatorname{span}\\{(1,1,1),(0,1,1)\\}, \mathbf{x}=(-1,2,1)\) b. \(U=\operatorname{span}\\{(1,-1,0),(-1,0,1)\\}, \mathbf{x}=(2,1,0)\) c. \(U=\operatorname{span}\\{(1,0,1,0),(1,1,1,0),(1,1,0,0)\\},\) \(\quad \mathbf{x}=(2,0,-1,3)\) d. \(U=\operatorname{span}\\{(1,-1,0,1),(1,1,0,0),(1,1,0,1)\\},\) \(\quad \mathbf{x}=(2,0,3,1)\)

Short answer

Follow the Gram-Schmidt process and projection steps to find orthogonal bases and closest vectors in subspaces for each case as explained above.

Step-by-step solution

Calculate Orthogonal Basis for Case a

Given the set \(U = \operatorname{span} \{(1,1,1), (0,1,1)\}\), we apply the Gram-Schmidt process. Start by taking \(\mathbf{v_1} = (1,1,1)\). Next, calculate \(\mathbf{v_2'} = (0,1,1) - \text{proj}_{\mathbf{v_1}}(0,1,1)\). This involves finding the projection: \[ \text{proj}_{\mathbf{v_1}}(0,1,1) = \frac{(0,1,1) \cdot (1,1,1)}{(1,1,1) \cdot (1,1,1)}(1,1,1) = \frac{2}{3}(1,1,1) = \left(\frac{2}{3}, \frac{2}{3}, \frac{2}{3}\right) \] Now calculate \(\mathbf{v_2'} = (0,1,1) - \left(\frac{2}{3}, \frac{2}{3}, \frac{2}{3}\right) = \left(-\frac{2}{3}, \frac{1}{3}, \frac{1}{3}\right)\). The orthogonal basis is \[ \{(1,1,1), (-\frac{2}{3}, \frac{1}{3}, \frac{1}{3})\} \].

Closest Vector in U for Case a

Find the closest vector in \(U\) to \(\mathbf{x} = (-1,2,1)\) by projecting \(\mathbf{x}\) onto \(\text{span}(\mathbf{v_1}, \mathbf{v_2'})\). First project \(\mathbf{x}\) onto \(\mathbf{v_1}\): \[ \text{proj}_{\mathbf{v_1}}(-1,2,1) = \frac{(-1,2,1) \cdot (1,1,1)}{(1,1,1) \cdot (1,1,1)}(1,1,1) = \frac{2}{3} (1,1,1) = \left(\frac{2}{3}, \frac{2}{3}, \frac{2}{3}\right) \] Next, project \(\mathbf{x}\) onto \(\mathbf{v_2'}\): \[ \text{proj}_{\mathbf{v_2'}}(-1,2,1) = \frac{(-1,2,1) \cdot (-\frac{2}{3}, \frac{1}{3}, \frac{1}{3})}{(-\frac{2}{3}, \frac{1}{3}, \frac{1}{3}) \cdot (-\frac{2}{3}, \frac{1}{3}, \frac{1}{3})} \left(-\frac{2}{3}, \frac{1}{3}, \frac{1}{3}\right) = \frac{\frac{5}{3}}{\frac{2}{3}}(-\frac{2}{3}, \frac{1}{3}, \frac{1}{3})\] Therefore, the vector closest to \(\mathbf{x}\) in \(U\) is the sum: \[ \left(\frac{2}{3}, \frac{2}{3}, \frac{2}{3}\right) + \left(-\frac{5}{3}, \frac{5}{6}, \frac{5}{6}\right) = \left(-\frac{1}{3}, \frac{3}{2}, \frac{3}{2}\right) \].

Calculate Orthogonal Basis for Case b

Here, \(U = \operatorname{span} \{(1,-1,0), (-1,0,1)\}\). We start \(\mathbf{v_1} = (1,-1,0)\). Use Gram-Schmidt to find \(\mathbf{v_2'} = (-1,0,1) - \text{proj}_{\mathbf{v_1}}(-1,0,1)\). Projection: \[ \text{proj}_{\mathbf{v_1}}(-1,0,1) = \frac{(-1,0,1) \cdot (1,-1,0)}{(1,-1,0) \cdot (1,-1,0)}(1,-1,0) = \frac{-1}{2}(1,-1,0) = \left(-\frac{1}{2}, \frac{1}{2}, 0\right) \] Calculate \(\mathbf{v_2'} = (-1,0,1) - \left(-\frac{1}{2}, \frac{1}{2}, 0\right) = \left(-\frac{1}{2}, -\frac{1}{2}, 1\right)\). The orthogonal set is \[ \{(1,-1,0), (-\frac{1}{2}, -\frac{1}{2}, 1)\} \].

Closest Vector in U for Case b

For \(\mathbf{x} = (2,1,0)\), find projections onto the orthogonal basis. Starting with \(\mathbf{v_1}\): \[ \text{proj}_{\mathbf{v_1}}(2,1,0) = \frac{(2,1,0) \cdot (1,-1,0)}{(1,-1,0) \cdot (1,-1,0)}(1,-1,0) = \frac{1}{2}(1,-1,0) = \left(\frac{1}{2}, -\frac{1}{2}, 0\right) \] Next, for \(\mathbf{v_2'}\): \[ \text{proj}_{\mathbf{v_2'}}(2,1,0) = \frac{(2,1,0) \cdot (-\frac{1}{2}, -\frac{1}{2}, 1)}{(-\frac{1}{2}, -\frac{1}{2}, 1) \cdot (-\frac{1}{2}, -\frac{1}{2}, 1)} \left(-\frac{1}{2}, -\frac{1}{2}, 1\right) = \ldots = \left(\frac{1}{3}, \frac{1}{3}, \frac{1}{3}\right) \] Closest vector is: \[ \left(\frac{1}{2}, -\frac{1}{2}, 0\right) + \left(\frac{1}{3}, \frac{1}{3}, \frac{1}{3}\right) \].

Calculate Orthogonal Basis for Case c

Given \(U = \operatorname{span}\{(1,0,1,0),(1,1,1,0),(1,1,0,0)\}\), use \(\mathbf{v_1} = (1,0,1,0)\). Find \(\mathbf{v_2'} = (1,1,1,0) - \text{proj}_{\mathbf{v_1}}(1,1,1,0)\), then continue with \(\mathbf{v_3'} = (1,1,0,0) - \text{proj}_{\text{span}(\mathbf{v_1},\mathbf{v_2'})}(1,1,0,0)\). Calculate projections and subtract to get \(\mathbf{v_2'}\) and \(\mathbf{v_3'}\). The orthogonal basis will be \[ \{\mathbf{v_1}, \mathbf{v_2'}, \mathbf{v_3'}\} \]. Calculations similar to previous steps.

Closest Vector in U for Case c

Given \(\mathbf{x} = (2,0,-1,3)\), project \(\mathbf{x}\) onto each orthogonal basis vector \(\mathbf{v_1}, \mathbf{v_2'}, \mathbf{v_3'}\), sum the projections to find the closest vector in \(U\). Project onto each and add: \(\text{proj}_{\text{span}(\text{all})}(\mathbf{x})\).

Calculate Orthogonal Basis for Case d

With \(U = \operatorname{span}\{(1,-1,0,1),(1,1,0,0),(1,1,0,1)\}\), set \(\mathbf{v_1} = (1,-1,0,1)\). Find \(\mathbf{v_2'} = (1,1,0,0) - \text{proj}_{\mathbf{v_1}}(1,1,0,0)\), then \(\mathbf{v_3'} = (1,1,0,1) - \text{proj}_{\text{span}(\mathbf{v_1},\mathbf{v_2'})}(1,1,0,1)\). Use projection formulas to compute \(\mathbf{v_2'}\) and \(\mathbf{v_3'}\). The orthogonal set is \[ \{\mathbf{v_1}, \mathbf{v_2'}, \mathbf{v_3'}\} \].

Closest Vector in U for Case d

For \(\mathbf{x} = (2,0,3,1)\), project \(\mathbf{x}\) onto the orthogonal basis vectors and sum the projections. Begin with \(\text{proj}_{\mathbf{v_1}}(\mathbf{x})\), \(\text{proj}_{\mathbf{v_2'}}(\mathbf{x})\), \(\text{proj}_{\mathbf{v_3'}}(\mathbf{x})\), and add them to find the closest vector.

Key concepts

Orthogonal Basis

An orthogonal basis for a subspace is a set of vectors that are both mutually perpendicular and span the subspace. This means each pair of vectors in the basis is orthogonal, and any vector in the subspace can be expressed as a linear combination of these basis vectors. Finding an orthogonal basis simplifies many vector calculations, including projections and computations involving dot products.

The Gram-Schmidt process is a reliable method to convert any set of linearly independent vectors into an orthogonal basis. The steps involve taking each vector in the original set and orthogonalizing it against all previously obtained orthogonal vectors using vector projections. The result is a set of vectors that still spans the subspace but with the added benefit of orthogonality.

Orthogonal bases are especially beneficial in simplifying the mathematics involved in vector space manipulation as they help avoid complications that arise from non-orthogonal vectors, particularly in projections.

Vector Projection

Vector projection is a way of projecting one vector onto another, resulting in a new vector that lies on the line of the vector being projected onto. This is especially useful in the Gram-Schmidt algorithm when orthogonalizing vectors.

Mathematically, the projection of a vector \(\mathbf{a}\) onto \(\mathbf{b}\) is given by:\[\text{proj}_{\mathbf{b}}(\mathbf{a}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\mathbf{b} \cdot \mathbf{b}} \mathbf{b}\] Here, the dot product in the numerator measures how much \(a\) is in the direction of \(\mathbf{b}\), and the divisor normalizes it to the length of \(\mathbf{b}\). This ensures that the projection result stays within the span of \(\mathbf{b}\) and is directly aligned with it.

In essence, vector projection is used in orthogonalizing vectors and finding the closest points on lines or planes, making it invaluable in geometry and linear algebra.

Subspace Span

A subspace span refers to the set of all possible linear combinations of a given set of vectors. Essentially, it is the vector space that these combinations fill.

For a subspace defined by vectors \(\{\mathbf{v_1}, \mathbf{v_2}, \ldots, \mathbf{v_n}\}\), the span is expressed as:\[\text{span}\{\mathbf{v_1}, \mathbf{v_2}, \ldots, \mathbf{v_n}\} = \{a_1\mathbf{v_1} + a_2\mathbf{v_2} + \ldots + a_n\mathbf{v_n} \mid a_i \in \mathbb{R}\}\]Understanding the concept of span is fundamental in linear algebra, as it helps visualize the scope or extent of a set of vectors, ensuring that if vectors span a space, any vector within that space can be fully represented by these vectors.

The Gram-Schmidt process ensures orthogonality while maintaining the same span, thus providing simplicity without changing the covered subspace.

Closest Vector in Subspace

Finding the closest vector in a subspace to a given vector involves projecting the vector onto the subspace. The resulting vector is the one within the subspace that is nearest to the given vector in terms of Euclidean distance.

For a vector \(\mathbf{x}\) projected onto a subspace \(U\) with an orthogonal basis \(\{\mathbf{v_1}, \mathbf{v_2}, \ldots\}\), the closest vector is derived by summing the projections of \(\mathbf{x}\) onto each basis vector:\[\text{proj}_{U}(\mathbf{x}) = \text{proj}_{\mathbf{v_1}}(\mathbf{x}) + \text{proj}_{\mathbf{v_2}}(\mathbf{x}) + \ldots\]This results in the vector in \(U\) that is nearest to \(\mathbf{x}\) by minimizing the orthogonal distance between them.

This concept is pivotal in applications like data approximation and projection techniques in higher-dimensional spaces, serving as the foundational step in reducing dimensionality or extracting feature vectors.