Question

Suppose the x-coordinates of the data \(\left( {{x_1},{y_1}} \right), \ldots ,\left( {{x_n},{y_n}} \right)\) are in mean deviation form, so that \(\sum {{x_i}} = 0\). Show that if \(X\) is the design matrix for the least-squares line in this case, then \({X^T}X\) is a diagonal matrix.

Short answer

It is verified that, \({X^T}X = \left( {\begin{aligned}n&0\\0&{\sum {{x^2}} }\end{aligned}} \right)\) is a diagonal matrix.

Step-by-step solution

The General Linear Model

The equation of the general linear model is given as:

\({\bf{y}} = X\beta + \in \)

Here, \({\bf{y}} = \left( {\begin{aligned}{{y_1}}\\{{y_2}}\\ \vdots \\{{y_n}}\end{aligned}} \right)\) is an observational vector, \(X = \left( {\begin{aligned}1&{{x_1}}& \cdots &{x_1^n}\\1&{{x_2}}& \cdots &{x_2^n}\\ \vdots & \vdots & \ddots & \vdots \\1&{{x_n}}& \cdots &{x_n^n}\end{aligned}} \right)\) is the design matrix, \(\beta = \left( {\begin{aligned}{{\beta _1}}\\{{\beta _2}}\\ \vdots \\{{\beta _n}}\end{aligned}} \right)\) is parameter vector, and \( \in = \left( {\begin{aligned}{{ \in _1}}\\{{ \in _2}}\\ \vdots \\{{ \in _n}}\end{aligned}} \right)\) is a residual vector.

Find design matrix, observation vector, parameter vector for given data

The given data points are:\(\left( {{x_1},{y_1}} \right), \ldots ,\left( {{x_n},{y_n}} \right)\).

Write the design matrix and observational vector for the given data points.

Design matrix: \(X = \left( {\begin{aligned}1&{{x_1}}\\1&{{x_2}}\\ \vdots & \vdots \\1&{{x_n}}\end{aligned}} \right)\)

Observational matrix: \({\bf{y}} = \left( {\begin{aligned}{{y_1}}\\{{y_2}}\\ \vdots \\{{y_n}}\end{aligned}} \right)\)

And the parameter vectorfor the given equation is,

\({\bf{\beta }} = \left( {\begin{aligned}{{\beta _0}}\\{{\beta _1}}\end{aligned}} \right)\)

Find \({X^T}X\)

Find\({X^T}X\).

\(\begin{aligned}{X^T}X &= {\left( {\begin{aligned}1&{{x_1}}\\1&{{x_2}}\\ \vdots & \vdots \\1&{{x_n}}\end{aligned}} \right)^T}\left( {\begin{aligned}1&{{x_1}}\\1&{{x_2}}\\ \vdots & \vdots \\1&{{x_n}}\end{aligned}} \right)\\ &= \left( {\begin{aligned}1& \cdots &1\\{{x_1}}& \cdots &{{x_n}}\end{aligned}} \right)\left( {\begin{aligned}1&{{x_1}}\\1&{{x_2}}\\ \vdots & \vdots \\1&{{x_n}}\end{aligned}} \right)\\ &= \left( {\begin{aligned}n&{\sum x }\\{\sum x }&{\sum {{x^2}} }\end{aligned}} \right)\end{aligned}\)

Hence, the matrix for \({X^T}X\) is \(\left( {\begin{aligned}n&{\sum x }\\{\sum x }&{\sum {{x^2}} }\end{aligned}} \right)\).

Check whether \({X^T}X\) is a diagonal matrix or not

As, \({X^T}X = \left( {\begin{aligned}n&{\sum x }\\{\sum x }&{\sum {{x^2}} }\end{aligned}} \right)\). It is given that \(\sum {{x_i}} = 0\), then,

\({X^T}X = \left( {\begin{aligned}n&0\\0&{\sum {{x^2}} }\end{aligned}} \right)\)

It can be seen that the non-diagonal elements are 0, so is a diagonal matrix.