Question

Suppose \(\mathbf{X}\) is an \(n \times p\) matrix with rank \(p\). (a) Show that \(\operatorname{ker}\left(\mathbf{X}^{\prime} \mathbf{X}\right)=\operatorname{ker}(\mathbf{X})\). (b) Use part (a) and the last exercise to show that if \(\mathbf{X}\) has full column rank, then \(\mathbf{X}^{\prime} \mathbf{X}\) is nonsingular.

Short answer

Part (a) is demonstrated by proving that for any vector in the kernel of \(\mathbf{X}^{\prime} \mathbf{X}\), it must also be in the kernel of \(\mathbf{X}\), and vice versa, thus showing that \(\operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X}) = \operatorname{ker}(\mathbf{X})\). In part (b), using the result from part (a), we know that the size of \(\operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X})\) is \(0\) if \(\mathbf{X}\) has full column rank. Thus, \(\mathbf{X}^{\prime} \mathbf{X}\) is nonsingular.

Step-by-step solution

Defining the Kernels

Let's consider two vectors \(v\) such that \(v \in \operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X})\) and \(v \in \operatorname{ker}(\mathbf{X})\). Then, by definition of a kernel, we have \(\mathbf{X}^{\prime} \mathbf{X} v = 0\) and \(\mathbf{X} v = 0\).

Proving \(\operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X}) \subseteq \operatorname{ker}(\mathbf{X})\)

For any \(v \in \operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X})\), we have \(v^{\prime} \mathbf{X}^{\prime} \mathbf{X} v = 0\). This implies \( (\mathbf{X} v)^{\prime} (\mathbf{X} v) = 0\). Since this holds only if \(\mathbf{X} v = 0\), we can conclude that \(v \in \operatorname{ker}(\mathbf{X})\).

Proving \(\operatorname{ker}(\mathbf{X}) \subseteq \operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X})\)

For any \(v \in \operatorname{ker}(\mathbf{X})\), we have \(\mathbf{X} v = 0\). This implies \(\mathbf{X}^{\prime} \mathbf{X} v = \mathbf{X}^{\prime} 0 = 0\). Thus, \(v \in \operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X})\).

Using part (a) to prove part (b)

From part (a), we know \(\operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X}) = \operatorname{ker}(\mathbf{X})\). Since \(\mathbf{X}\) has full column rank \(p\), the size of its kernel (\(\operatorname{ker}(\mathbf{X})\)) is \(0\). Therefore, the size of \(\operatorname{ker}(\mathbf{X}^{\prime} \mathbf{X})\) is also \(0\), which means that \(\mathbf{X}^{\prime} \mathbf{X}\) is nonsingular.