Question

Let \(X\) be \(m \times n\) data matrix such that \({X^T}X\) is invertible., and let \(M = {I_m} - X{\left( {{X^T}X} \right)^{ - {\bf{1}}}}{X^T}\). Add a column \({x_{\bf{0}}}\) to the data and form

\(W = \left[ {\begin{array}{*{20}{c}}X&{{x_{\bf{0}}}}\end{array}} \right]\)

Compute \({W^T}W\). The \(\left( {{\bf{1}},{\bf{1}}} \right)\) entry is \({X^T}X\). Show that the Schur complement (Exercise 15) of \({X^T}X\) can be written in the form \({\bf{x}}_{\bf{0}}^TM{{\bf{x}}_{\bf{0}}}\). It can be shown that the quantity \({\left( {{\bf{x}}_{\bf{0}}^TM{{\bf{x}}_{\bf{0}}}} \right)^{ - {\bf{1}}}}\) is the \(\left( {{\bf{2}},{\bf{2}}} \right)\)-entry in \({\left( {{W^T}W} \right)^{ - {\bf{1}}}}\). This entry has a useful statistical interpretation, under appropriate hypotheses.

In the study of engineering control of physical systems, a standard set of differential equations is transformed by Laplace transforms into the following system of linear equations:

\(\left[ {\begin{array}{*{20}{c}}{A - s{I_n}}&B\\C&{{I_m}}\end{array}} \right]\left[ {\begin{array}{*{20}{c}}{\bf{x}}\\{\bf{u}}\end{array}} \right] = \left[ {\begin{array}{*{20}{c}}{\bf{0}}\\{\bf{y}}\end{array}} \right]\)

Where \(A\) is \(n \times n\), \(B\) is \(n \times m\), \(C\) is \(m \times n\), and \(s\) is a variable. The vector \({\bf{u}}\) in \({\mathbb{R}^m}\) is the “input” to the system, \({\bf{y}}\) in \({\mathbb{R}^m}\) is the “output” and \({\bf{x}}\) in \({\mathbb{R}^n}\) is the “state” vector. (Actually, the vectors \({\bf{x}}\), \({\bf{u}}\) and \({\bf{v}}\) are functions of \(s\), but we suppress this fact because it does not affect the algebraic calculations in Exercises 19 and 20.)

Short answer

\({\bf{x}}_0^TM{{\bf{x}}_0}\)

Step-by-step solution

Find the value of \({W^T}W\)

Given, \(W = \left[ {\begin{array}{*{20}{c}}X&{{{\bf{x}}_0}}\end{array}} \right]\).

Then,

\(\begin{array}{c}{W^T}W = \left[ {\begin{array}{*{20}{c}}{{X^T}}\\{{\bf{x}}_0^T}\end{array}} \right]\left[ {\begin{array}{*{20}{c}}X&{{{\bf{x}}_0}}\end{array}} \right]\\ = \left[ {\begin{array}{*{20}{c}}{{X^T}X}&{{X^T}{{\bf{x}}_0}}\\{{\bf{x}}_0^TX}&{{\bf{x}}_0^T{{\bf{x}}_0}}\end{array}} \right].\end{array}\)

Find the Schur complement \(S\)

By the formula \(S\)(Schur complement) from Exercise 15,

\(\begin{array}{c}S = {\bf{x}}_0^T{{\bf{x}}_0} - {\bf{x}}_0^TX{\left( {{X^T}X} \right)^{ - 1}}{X^T}{{\bf{x}}_0}\\ = {\bf{x}}_0^T\left( {{I_m} - X{{\left( {{X^T}X} \right)}^{ - 1}}{X^T}} \right){{\bf{x}}_0}\\ = {\bf{x}}_0^TM{{\bf{x}}_0}.\end{array}\)

So, the value of \(S\) is \({\bf{x}}_0^TM{{\bf{x}}_0}\).