Question

Suppose that columns of A are linearly independent. Determine what happens to the least-square solution\(\widehat {\bf{x}}\)of\(A{\bf{x}} = {\mathop{\rm b}\nolimits} \)when b is replaced by\(c{\bf{b}}\)for some nonzero scalar\(c\).

Short answer

The least-square solution \(\widehat {\bf{x}}\) of \(A{\bf{x}} = {\bf{b}}\) is \({c_1}{\widehat {\bf{x}}_1} + {c_2}{\widehat {\bf{x}}_2}\).

Step-by-step solution

Statement in Theorem 14 

Consider \(A\) as an \(m \times n\) matrix. Then, the following statement is equivalent.

1. The equation \(A{\bf{x}} = {\bf{b}}\) contains a uniqueleast-square solutionfor every \({\bf{b}}\) in \({\mathbb{R}^m}\).
2. The columns of \(A\) are known as linearly independent.
3. A matrix \({A^T}A\) is known as invertible.

If these statements are true, then the least-squares solution \(\widehat {\bf{x}}\) is provided by,

\(\widehat {\bf{x}} = {\left( {{A^T}A} \right)^{ - 1}}{A^T}{\bf{b}}\) … (1)

Determine what happens to the least-square solution \(\widehat {\bf{x}}\) of \(A{\bf{x}} = {\mathop{\rm b}\nolimits} \)

According to theorem 14, the equation \(A{\bf{x}} = {\bf{b}}\) contains a unique least-square for every \({\bf{b}}\) in \({\mathbb{R}^m}\), when \(c \ne 0\), then the least-square solution of \(A{\bf{x}} = c{\bf{b}}\) is provided by \(\widehat {\bf{x}} = {\left( {{A^T}A} \right)^{ - 1}}{A^T}{\bf{b}}\).

Consider \({\bf{b}} = {c_1}{{\bf{b}}_1} + {c_2}{{\bf{b}}_2}\) and use the linearity of matrix multiplication to obtain the value as shown below:

\(\begin{array}{c}{\left( {{A^T}A} \right)^{ - 1}}{A^T}\left( {{c_1}{{\bf{b}}_1} + {c_2}{{\bf{b}}_2}} \right) = {c_1}{\left( {{A^T}A} \right)^{ - 1}}{A^T}{{\bf{b}}_1} + {c_2}{\left( {{A^T}A} \right)^{ - 1}}{A^T}{{\bf{b}}_2}\\ = {c_1}{\widehat {\bf{x}}_1} + {c_2}{\widehat {\bf{x}}_2}\end{array}\)

Thus, the least-square solution \(\widehat {\bf{x}}\) of \(A{\bf{x}} = {\bf{b}}\) is \({c_1}{\widehat {\bf{x}}_1} + {c_2}{\widehat {\bf{x}}_2}\).