Question

Question: Let \({\bf{X}}\) denote a vector that varies over the columns of a \(p \times N\) matrix of observations, and let \(P\) be a \(p \times p\) orthogonal matrix. Show that the change of variable \({\bf{X}} = P{\bf{Y}}\) does not change the total variance of the data. (Hint: By Exercise 11, it suffices to show that \(tr\left( {{P^T}SP} \right) = tr\left( S \right)\). Use a property of the trace mentioned in Exercise 25 in Section 5.4.)

Short answer

It is verified that the total variance would not change when variables change as \({\bf{X}} = P{\bf{Y}}\).

Step-by-step solution

Mean Deviation form and Covariance Matrix.

The Mean Deviation formof any \(p \times N\)is given by:

\(B = \left( {\begin{array}{*{20}{c}}{{{{\bf{\hat X}}}_1}}&{{{{\bf{\hat X}}}_2}}&{........}&{{{{\bf{\hat X}}}_N}}\end{array}} \right)\)

Whose \(p \times p\) covariance matrixis:

\(S = \frac{1}{{N - 1}}B{B^T}\)

The Variance

From exercise 11,we have:

\({S_Y} = {P^T}SP\)

When variable changes as:\({\bf{X}} = P{\bf{Y}}\)

The traces of the covariance matrices\({S_Y}{\rm{ and }}S\)will be the same.

The total variance of the data is given by\({\bf{Y}}\)is\({\rm{tr}}\left( {{P^T}SP} \right)\).

For two similar matrices\(A,B\)are such that,\({\bf{B}} = P{\bf{A}}{P^{ - 1}}\)which implies\({\rm{tr}}\left( {\bf{B}} \right) = {\rm{tr}}\left( {P{\bf{A}}{P^{ - 1}}} \right)\).

In the obtained equation, if\(P\)is an orthogonal matrix, then\({P^T} = {P^{ - 1}}\).

Apply trace on both sides of\({P^T} = {P^{ - 1}}\)and simplify.

\(\begin{array}{c}{\rm{tr}}\left( {{P^T}SP} \right) = {\rm{tr}}\left( {{P^{ - 1}}SP} \right)\\ = {\rm{tr}}\left( {{P^{ - 1}}PS} \right)\\ = {\rm{tr}}\left( {\left( {{P^{ - 1}}P} \right)S} \right)\\ = {\rm{tr}}\left( {IS} \right)\\ = {\rm{tr}}\left( S \right)\end{array}\)

Thus, the total variance would not changewhen variables change as:\({\bf{X}} = P{\bf{Y}}\).

Hence, this is the required proof.