Question

Question: 13. The sample covariance matrix is a generalization of a formula for the variance of a sample of \(N\) scalar measurements, say \({t_1},................,{t_N}\). If \(m\) is the average of \({t_1},................,{t_N}\), then the sample variance is given by

\(\frac{1}{{N - 1}}\sum\limits_{k = 1}^n {{{\left( {{t_k} - m} \right)}^2}} \)

Show how the sample covariance matrix, \(S\), defined prior to Example 3, may be written in a form similar to (1). (Hint: Use partitioned matrix multiplication to write \(S\) as \(\frac{1}{{N - 1}}\) times the sum of \(N\) matrices of size \(p \times p\). For \(1 \le k \le N\), write \({X_k} - M\) in place of \({\hat X_k}\).)

Short answer

It is verified that the sample covariance matrix can be written in the form:

\(S = \frac{1}{{N - 1}}\sum\limits_{k = 1}^N {\left( {{X_k} - M} \right){{\left( {{X_k} - M} \right)}^T}} \)

Step-by-step solution

Mean Deviation form and Covariance Matrix.

The Mean Deviation formof any \(p \times N\)is given by:

\(B = \left( {\begin{array}{*{20}{c}}{{{{\bf{\hat X}}}_1}}&{{{{\bf{\hat X}}}_2}}&{........}&{{{{\bf{\hat X}}}_N}}\end{array}} \right)\)

Whose \(p \times p\) covariance matrixis:

\(S = \frac{1}{{N - 1}}B{B^T}\)

The Mean Deviation Form

Let the sample mean be\(M\). Then,we have:

\({{\bf{\hat X}}_k} = {{\bf{X}}_k} - M\)

The mean deviation formis given by:

\(B = \left( {\begin{array}{*{20}{c}}{{{{\bf{\hat X}}}_1}}&{......}&{{{{\bf{\hat X}}}_N}}\end{array}} \right)\)

Now, thecovariance matrixcan be given as:

\(\begin{array}{c}S = \frac{1}{{N - 1}}B{B^T}\\ = \frac{1}{{N - 1}}\left( {\begin{array}{*{20}{c}}{{{{\bf{\hat X}}}_1}}&{......}&{{{{\bf{\hat X}}}_N}}\end{array}} \right){\left( {\begin{array}{*{20}{c}}{{{{\bf{\hat X}}}_1}}&{......}&{{{{\bf{\hat X}}}_N}}\end{array}} \right)^T}\\ = \frac{1}{{N - 1}}\sum\limits_{k = 1}^N {\left( {{{{\bf{\hat X}}}_k}} \right){{\left( {{{{\bf{\hat X}}}_k}} \right)}^T}} \\ = \frac{1}{{N - 1}}\sum\limits_{k = 1}^N {\left( {{{\bf{X}}_k} - M} \right){{\left( {{{\bf{X}}_k} - M} \right)}^T}} \end{array}\)

Hence, this is the required proof.