Often in regression the mean of the random variable \(Y\) is a linear function
of \(p\) -values \(x_{1}, x_{2}, \ldots, x_{p}\), say \(\beta_{1} x_{1}+\beta_{2}
x_{2}+\cdots+\beta_{p} x_{p}\), where \(\beta^{\prime}=\left(\beta_{1},
\beta_{2}, \ldots, \beta_{p}\right)\)
are the regression coefficients. Suppose that \(n\) values,
\(\boldsymbol{Y}^{\prime}=\left(Y_{1}, Y_{2}, \ldots, Y_{n}\right)\), are
observed for the \(x\) -values in \(\boldsymbol{X}=\left[x_{i j}\right]\), where
\(\boldsymbol{X}\) is an \(n \times p\) design matrix and its ith row is
associated with \(Y_{i}, i=1,2, \ldots, n\). Assume that \(\boldsymbol{Y}\) is
multivariate normal with mean \(\boldsymbol{X} \boldsymbol{\beta}\) and
variance-covariance matrix \(\sigma^{2} \boldsymbol{I}\), where \(\boldsymbol{I}\)
is the \(n \times n\) identity matrix.
(a) Note that \(Y_{1}, Y_{2}, \ldots, Y_{n}\) are independent. Why?
(b) Since \(\boldsymbol{Y}\) should approximately equal its mean \(\boldsymbol{X}
\boldsymbol{\beta}\), we estimate \(\boldsymbol{\beta}\) by solving the normal
equations \(\boldsymbol{X}^{\prime} \boldsymbol{Y}=\boldsymbol{X}^{\prime}
\boldsymbol{X} \boldsymbol{\beta}\) for \(\boldsymbol{\beta}\). Assuming that
\(\boldsymbol{X}^{\prime} \boldsymbol{X}\) is non-
singular, solve the equations to get
\(\hat{\beta}=\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1}
\boldsymbol{X}^{\prime} \boldsymbol{Y}\). Show that \(\hat{\boldsymbol{\beta}}\)
has a
multivariate normal distribution with mean \(\boldsymbol{\beta}\) and variance-
covariance matrix
$$
\sigma^{2}\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1}
$$
(c) Show that
$$
(\boldsymbol{Y}-\boldsymbol{X}
\boldsymbol{\beta})^{\prime}(\boldsymbol{Y}-\boldsymbol{X}
\boldsymbol{\beta})=(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta})^{\prime}\left(\boldsymbol{X}^{\prime}
\boldsymbol{X}\right)(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta})+(\boldsymbol{Y}-\boldsymbol{X}
\hat{\boldsymbol{\beta}})^{\prime}(\boldsymbol{Y}-\boldsymbol{X}
\hat{\boldsymbol{\beta}})
$$
For the remainder of the exercise, let \(Q\) denote the quadratic form on the
left side of this expression and \(Q_{1}\) and \(Q_{2}\) denote the respective
quadratic forms on the right side. Hence, \(Q=Q_{1}+Q_{2}\).
(d) Show that \(Q_{1} / \sigma^{2}\) is \(\chi^{2}(p)\).
(e) Show that \(Q_{1}\) and \(Q_{2}\) are independent.
(f) Argue that \(Q_{2} / \sigma^{2}\) is \(\chi^{2}(n-p)\).
(g) Find \(c\) so that \(c Q_{1} / Q_{2}\) has an \(F\) -distribution.
(h) The fact that a value \(d\) can be found so that \(P\left(c Q_{1} / Q_{2}
\leq d\right)=1-\alpha\) could be used to find a \(100(1-\alpha) \%\) confidence
ellipsoid for \(\boldsymbol{\beta} .\) Explain.
