Question

Find the least square solutions of the system $\mathbf{A}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{x}}\mathbf{=}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{b}}$where

$\mathbf{A}\mathbf{=}\left[\begin{array}{cc}1& 1\\ {10}^{-10}& 0\\ 0& {10}^{-10}\end{array}\right]\mathbf{}\mathbf{and}\mathbf{}\stackrel{\mathbf{\rightharpoonup }}{\mathbf{b}}\mathbf{=}\left[\begin{array}{c}1\\ {10}^{-10}\\ {10}^{-10}\end{array}\right]$.

Short answer

The solution is $a=\left(\frac{1-{10}^{-40}}{{10}^{-40}+{10}^{-20}-1}\right),andb=\frac{{10}^{-20}\left(1+{10}^{-20}\right)}{\left({10}^{-40}+{10}^{-20}-1\right)}$.

Step-by-step solution

Step:1 Definition of least square

Consider a linear system as $\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}=\stackrel{\rightharpoonup }{\mathrm{b}}$.

Here A is an matrix, a vector $\stackrel{\rightharpoonup }{\mathrm{x}}$in ${\mathrm{R}}^{\mathrm{n}}$called a least square solution of this system if role="math" localid="1659687531916" $||\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}\stackrel{\rightharpoonup }{{\mathrm{x}}^{\mathrm{k}}}||\le ||\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}\stackrel{\rightharpoonup }{{\mathrm{x}}^{\mathrm{k}}}||\mathrm{for}\mathrm{all}\stackrel{\rightharpoonup }{\mathrm{x}}\mathrm{in}{\mathrm{R}}^{\mathrm{m}}$.

Step:2 Explanation of the solution

Consider a linear system as $\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}=\stackrel{\rightharpoonup }{\mathrm{b}}$where $\mathrm{A}=\left[\begin{array}{cc}1& 1\\ {10}^{-10}& 0\\ 0& {10}^{-10}\end{array}\right]\mathrm{and}\stackrel{\rightharpoonup }{\mathrm{b}}=\left[\begin{array}{c}1\\ {10}^{-10}\\ {10}^{-10}\end{array}\right]$and A also A is A is an matrix, a vector $\stackrel{\rightharpoonup }{\mathrm{x}}$in ${\mathrm{R}}^{\mathrm{n}}$called a least square solution of this system if

localid="1659688022040" $||\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}{\stackrel{\rightharpoonup }{\mathrm{x}}}^{\mathrm{k}}||\le ||\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}{\stackrel{\rightharpoonup }{\mathrm{x}}}^{\mathrm{k}}||\mathrm{for}\mathrm{all}\stackrel{\rightharpoonup }{\mathrm{x}}\mathrm{in}{\mathrm{R}}^{\mathrm{m}}$

The term least square solution reflects the fact that minimizing the sum of the squares of the component of the vector $\left(\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}\right)$.

To show ${\stackrel{\rightharpoonup }{\mathrm{x}}}^{\mathrm{k}}$of a linear system $\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}=\stackrel{\rightharpoonup }{\mathrm{b}}$.

Consider the following string of equivalent statements.

The vector ${\stackrel{\rightharpoonup }{\mathrm{x}}}^{\mathrm{k}}$is a least square solution of the system $\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}=\stackrel{\rightharpoonup }{\mathrm{b}}$.

Then by the definition as follows.

$\iff ||\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}{\stackrel{\rightharpoonup }{\mathrm{x}}}^{\mathrm{k}}||\le ||\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}{\stackrel{\rightharpoonup }{\mathrm{x}}}^{\mathrm{k}}||\mathrm{for}\mathrm{all}\stackrel{\rightharpoonup }{\mathrm{x}}\mathrm{in}{\mathrm{R}}^{\mathrm{m}}$

Also by the theorem 5.4.3 as follows.

$\iff \mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}=\mathrm{proj}\left(\stackrel{\rightharpoonup }{\mathrm{b}}\right)\mathrm{where}\mathrm{V}=\mathrm{im}\left(\mathrm{A}\right)$

Similarly by the theorem 5.1.4 and 5.4.1 as follows.

$$\begin{gathered} \iff \stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}\mathrm{is}\mathrm{in}\mathrm{V}-={\left(\mathrm{imA}\right)}^{\perp }=\ker \left({\mathrm{A}}^{\mathrm{T}}\right) \\ \Updownarrow \\ {\mathrm{A}}^{\mathrm{T}}\left(\stackrel{\rightharpoonup }{\mathrm{b}}-\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}\right)=\stackrel{\rightharpoonup }{0} \\ \Updownarrow \\ {\mathrm{A}}^{\mathrm{T}}\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}={\mathrm{A}}^{\mathrm{T}}\stackrel{\rightharpoonup }{\mathrm{b}} \end{gathered}$$.

Therefore, the least square of the system $\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}=\stackrel{\rightharpoonup }{\mathrm{b}}$are the exact solution of the system .

${\mathrm{A}}^{\mathrm{T}}\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}={\mathrm{A}}^{\mathrm{T}}\stackrel{\rightharpoonup }{\mathrm{b}}$

The system ${\mathrm{A}}^{\mathrm{T}}\mathrm{A}\stackrel{\rightharpoonup }{\mathrm{x}}={\mathrm{A}}^{\mathrm{T}}\stackrel{\rightharpoonup }{\mathrm{b}}$is called the normal equation of .

Now, simplify as follows.

$$\begin{gathered} \left[\begin{array}{ccc}1& {10}^{-10}& 0\\ 1& 0& {10}^{-10}\end{array}\right]\left[\begin{array}{cc}1& 1\\ {10}^{-10}& 0\\ 0& {10}^{-10}\end{array}\right]=\left[\begin{array}{ccc}1& {10}^{-10}& 0\\ 1& 0& {10}^{-10}\end{array}\right]\left[\begin{array}{c}1\\ {10}^{-10}\\ {10}^{-10}\end{array}\right] \\ \left[\begin{array}{cc}1+{10}^{10}& 1\\ 1& {10}^{-20}\end{array}\right]\stackrel{\rightharpoonup }{\mathrm{x}}=\left[\begin{array}{c}1+{10}^{-20}\\ 1+{10}^{-20}\end{array}\right] \\ \left[\begin{array}{c}{\mathrm{x}}_1\\ {\mathrm{x}}_2\end{array}\right]=\left[\begin{array}{c}\mathrm{a}\\ \mathrm{b}\end{array}\right] \\ \mathrm{where}{\mathrm{x}}_1=\mathrm{a}\left(\frac{1+{10}^{-40}}{{10}^{-40}+{10}^{-20}-1}\right)\mathrm{and}{\mathrm{x}}_2=\mathrm{b}=\frac{{10}^{-20}\left(1+{10}^{-20}\right)}{\left({10}^{-40}+{10}^{-20}-1\right)} \\ \mathrm{Hence},\mathrm{the}\mathrm{solution}\mathrm{is}\mathrm{a}=\left(\frac{1+{10}^{-40}}{{10}^{-40}+{10}^{-20}-1}\right)\mathrm{and}\mathrm{b}=\frac{{10}^{-20}\left(1+{10}^{-20}\right)}{\left({10}^{-40}+{10}^{-20}-1\right)}. \end{gathered}$$