Question

Suppose \(\boldsymbol{Y}\) is an \(n \times 1\) random vector, \(\boldsymbol{X}\) is an \(n \times p\) matrix of known constants of rank \(p\), and \(\beta\) is a \(p \times 1\) vector of regression coefficients. Let \(\boldsymbol{Y}\) have a \(N\left(\boldsymbol{X} \boldsymbol{\beta}, \sigma^{2} \boldsymbol{I}\right)\) distribution. Obtain the pdf of \(\hat{\boldsymbol{\beta}}=\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \boldsymbol{Y}\).

Short answer

The derived probability density function for \(\hat{\boldsymbol{\beta}}\) is a multivariate normal distribution. Its parameters depend on the matrix \(\boldsymbol{X}\) and the variance \(\sigma^2\).

Step-by-step solution

Define \(\hat{\boldsymbol{\beta}}\)

The estimator of the vector of regression coefficients, \(\hat{\boldsymbol{\beta}}\), is expressed as \((\boldsymbol{X}'\boldsymbol{X})^{-1}\boldsymbol{X}' \boldsymbol{Y}\). This formula is used to obtain the least squares estimate of the regression coefficients in a multiple regression model.

Express \(\boldsymbol{Y}\) in terms of \(\hat{\boldsymbol{\beta}}\)

Rearrange the equation for \(\hat{\boldsymbol{\beta}}\) to write \(\boldsymbol{Y}\) as \(\boldsymbol{Y} = \boldsymbol{X}(\boldsymbol{X}'\boldsymbol{X})\hat{\boldsymbol{\beta}}\). This relates the vector of observations to the vector of regression coefficients and the matrix of predictors.

Insert \(\boldsymbol{Y}\) into the distribution

The distribution for \(\boldsymbol{Y}\) is given as \(N(\boldsymbol{X}\boldsymbol{\beta}, \sigma^2 \boldsymbol{I})\). Substituting \(\boldsymbol{Y}\) as determined in Step 2 into this equation provides the joint distribution of \(\boldsymbol{Y}|X\).

Obtain the pdf

The distribution of the estimated regression coefficients, \(\hat{\boldsymbol{\beta}}\), is identified as the marginal distribution of \(\boldsymbol{Y}| \boldsymbol{X}\) with respect to \(\hat{\boldsymbol{\beta}}\) using standard properties of multivariate normal distributions, including that a linear transformation of a multivariate normal random vector is also normally distributed.

Determine the form of the pdf

Given that the original distribution of the random vector \(\boldsymbol{Y}\) was specified as a multivariate normal distribution, the derived pdf for \(\hat{\boldsymbol{\beta}}\) will also be normally distributed. Here, the parameters of this distribution will depend on the matrix \(\boldsymbol{X}\) and the variance \(\sigma^2\).