Warning: foreach() argument must be of type array|object, bool given in /var/www/html/web/app/themes/studypress-core-theme/template-parts/header/mobile-offcanvas.php on line 20

Often in regression the mean of the random variable \(Y\) is a linear function of \(p\) -values \(x_{1}, x_{2}, \ldots, x_{p}\), say \(\beta_{1} x_{1}+\beta_{2} x_{2}+\cdots+\beta_{p} x_{p}\), where \(\boldsymbol{\beta}^{\prime}=\left(\beta_{1}, \beta_{2}, \ldots, \beta_{p}\right)\) are the regression coefficients. Suppose that \(n\) values, \(\boldsymbol{Y}^{\prime}=\left(Y_{1}, Y_{2}, \ldots, Y_{n}\right)\) are observed for the \(x\) -values in \(\boldsymbol{X}=\left[x_{i j}\right]\), where \(\boldsymbol{X}\) is an \(n \times p\) design matrix and its ith row is associated with \(Y_{i}, i=1,2, \ldots, n .\) Assume that \(Y\) is multivariate normal with mean \(\boldsymbol{X} \boldsymbol{\beta}\) and variance-covariance matrix \(\sigma^{2} \boldsymbol{I}\), where \(\boldsymbol{I}\) is the \(n \times n\) identity matrix. (a) Note that \(Y_{1}, Y_{2}, \ldots, Y_{n}\) are independent. Why? (b) Since \(\boldsymbol{Y}\) should approximately equal its mean \(\boldsymbol{X} \boldsymbol{\beta}\), we estimate \(\boldsymbol{\beta}\) by solving the normal equations \(\boldsymbol{X}^{\prime} \boldsymbol{Y}=\boldsymbol{X}^{\prime} \boldsymbol{X} \boldsymbol{\beta}\) for \(\boldsymbol{\beta}\). Assuming that \(\boldsymbol{X}^{\prime} \boldsymbol{X}\) is non- singular, solve the equations to get \(\hat{\boldsymbol{\beta}}=\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \boldsymbol{Y}\). Show that \(\hat{\boldsymbol{\beta}}\) has a multivariate normal distribution with mean \(\boldsymbol{\beta}\) and variance-covariance matrix $$ \sigma^{2}\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} $$ (c) Show that $$ (\boldsymbol{Y}-\boldsymbol{X} \boldsymbol{\beta})^{\prime}(\boldsymbol{Y}-\boldsymbol{X} \boldsymbol{\beta})=(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta})^{\prime}\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta})+(\boldsymbol{Y}-\boldsymbol{X} \hat{\boldsymbol{\beta}})^{\prime}(\boldsymbol{Y}-\boldsymbol{X} \hat{\boldsymbol{\beta}}) $$ say \(Q=Q_{1}+Q_{2}\) for convenience. (d) Show that \(Q_{1} / \sigma^{2}\) is \(\chi^{2}(p)\). (e) Show that \(Q_{1}\) and \(Q_{2}\) are independent. (f) Argue that \(Q_{2} / \sigma^{2}\) is \(\chi^{2}(n-p)\). (g) Find \(c\) so that \(c Q_{1} / Q_{2}\) has an \(F\) -distribution. (h) The fact that a value \(d\) can be found so that \(P\left(c Q_{1} / Q_{2} \leq d\right)=1-\alpha\) could be used to find a \(100(1-\alpha)\) percent confidence ellipsoid for \(\beta\). Explain.

Short Answer

Expert verified
This exercise involves mathematical proofs related to several quantities derived from observations and predictors that are organized into a matrix in multiple linear regression. With normal distribution, identity covariance matrix and linear relationship between predictors and observations, we derived several important properties and relationships like normal distribution of estimates, independence of residuals, chi-square and F-distributions for squared quantities and statistical inference foundations for confidence intervals.

Step by step solution

01

Part (a)

Given that the variance-covariance matrix is \(\sigma^{2} \boldsymbol{I}\), where \(\boldsymbol{I}\) is the identity matrix, it implies all the off-diagonal elements representing the covariance of any two different \(Y\) values are zero. Thus, \(Y_{1}, Y_{2}, \ldots, Y_{n}\) are uncorrelated and since they are normal, they are also independent.
02

Part (b)

Given the normal equation \(\boldsymbol{X}^{\prime} \boldsymbol{Y}=\boldsymbol{X}^{\prime} \boldsymbol{X} \boldsymbol{\beta}\), we can solve for \(\boldsymbol{\beta}\) to get the least squares estimates \(\hat{\boldsymbol{\beta}}=\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \boldsymbol{Y}\). Under normal distribution assumption, the vector \(\boldsymbol{\beta}\) also has a multivariate normal distribution with mean \(\boldsymbol{\beta}\) and variance-covariance matrix \(\sigma^{2}\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1}\) from elements variance for multivariate normal distribution.
03

Part (c)

To prove the equation, expand both sides, as \((a-b)'(a-b) = a'a - 2 a'b + b'b\). After simplification, both sides will be equal.
04

Part (d)

The quantity \(Q_{1} / \sigma^{2} =(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta})^{\prime}\left(\boldsymbol{X}^{\prime}\boldsymbol{X}\right)(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta}) / \sigma^{2}\) follows a \(\chi^2\) distribution with \(p\) degrees of freedom by the definition of chi-square distribution and proved under (b)
05

Part (e)

The quantities \(Q_1\) and \(Q_2\) can be shown to be independent because they are functions of uncorrelated random variables.
06

Part (f)

The quantity \(Q_{2} / \sigma^{2} = (\boldsymbol{Y}-\boldsymbol{X}\hat{\boldsymbol{\beta}})^{\prime}(\boldsymbol{Y}-\boldsymbol{X}\hat{\boldsymbol{\beta}}) / \sigma^{2}\) follows a \(\chi^2\) distribution with \((n-p)\) degrees of freedom, by property of chi-square distribution where each of \(n-p\) residuals \((\boldsymbol{Y}-\boldsymbol{X} \hat{\boldsymbol{\beta}})\) squares contributes 1 degree.
07

Part (g)

To give \(c Q_{1} / Q_{2}\) an \(F\) -distribution, \(c\) must be the ratio of the degrees of freedom of \(Q_{1}\) and \(Q_{2}\), so \(c= p / (n-p)\) as they are the degrees of freedom from (d) and (f). An F-distribution describes ratio of two chi-square divided by their degrees of freedom.
08

Part (h)

Using \(Q_1\) and \(Q_2\) we can get an F distribution. We can also calculate a cut-off score \(d\) by defining our acceptable error to be \(\alpha\). Using the F-distribution table or function, we can find a value \(d\) that \[P\left(c Q_{1} / Q_{2} \leq d\right)=1-\alpha\]. This is the basis of statistical testing and establishing confidence intervals. We can obtain a \(100(1-\alpha)\%\) confidence ellipsoid for \(\beta\) by creating bounds using the relationship of \(\beta\) to the F-distribution.

Unlock Step-by-Step Solutions & Ace Your Exams!

  • Full Textbook Solutions

    Get detailed explanations and key concepts

  • Unlimited Al creation

    Al flashcards, explanations, exams and more...

  • Ads-free access

    To over 500 millions flashcards

  • Money-back guarantee

    We refund you if you fail your exam.

Over 30 million students worldwide already upgrade their learning with Vaia!

One App. One Place for Learning.

All the tools & learning materials you need for study success - in one app.

Get started for free

Most popular questions from this chapter

Let \(X_{1}, X_{2}, X_{3}\) be a random sample from the normal distribution \(N\left(0, \sigma^{2}\right)\). Are the quadratic forms \(X_{1}^{2}+3 X_{1} X_{2}+X_{2}^{2}+X_{1} X_{3}+X_{3}^{2}\) and \(X_{1}^{2}-2 X_{1} X_{2}+\frac{2}{3} X_{2}^{2}-\) \(2 X_{1} X_{2}-X_{3}^{2}\) independent or dependent?

Suppose \(\boldsymbol{Y}\) is an \(n \times 1\) random vector, \(\boldsymbol{X}\) is an \(n \times p\) matrix of known constants of rank \(p\), and \(\beta\) is a \(p \times 1\) vector of regression coefficients. Let \(\boldsymbol{Y}\) have a \(N\left(\boldsymbol{X} \boldsymbol{\beta}, \sigma^{2} \boldsymbol{I}\right)\) distribution. Discuss the joint pdf of \(\hat{\boldsymbol{\beta}}=\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \boldsymbol{Y}\) and \(\boldsymbol{Y}^{\prime}\left[\boldsymbol{I}-\boldsymbol{X}\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime}\right] \boldsymbol{Y} / \sigma^{2}\)

Let \(\mu_{1}, \mu_{2}, \mu_{3}\) be, respectively, the means of three normal distributions with a common but unknown variance \(\sigma^{2}\). In order to test, at the \(\alpha=5\) percent significance level, the hypothesis \(H_{0}: \mu_{1}=\mu_{2}=\mu_{3}\) against all possible alternative hypotheses, we take an independent random sample of size 4 from each of these distributions. Determine whether we accept or reject \(H_{0}\) if the observed values from these three distributions are, respectively, $$ \begin{array}{lrrrr} X_{1}: & 5 & 9 & 6 & 8 \\ X_{2}: & 11 & 13 & 10 & 12 \\ X_{3}: & 10 & 6 & 9 & 9 \end{array} $$

If \(A_{1}, A_{2}, \ldots, A_{k}\) are events, prove, by induction, Boole's inequality $$ P\left(A_{1} \cup A_{2} \cup \cdots \cup A_{k}\right) \leq \sum_{1}^{k} P\left(A_{i}\right) $$ Then show that $$ P\left(A_{1}^{c} \cap A_{2}^{c} \cap \cdots \cap A_{k}^{c}\right) \geq 1-\sum_{1}^{b} P\left(A_{i}\right) $$

Using the notation of Section 9.2, assume that the means \(\mu_{j}\) satisfy a linear function of \(j\), nanely \(\mu_{j}=c+d[j-(b+1) / 2] .\) Let independent random samples of size \(a\) be taken from the \(b\) normal distributions having means \(\mu_{1}, \mu_{2}, \ldots, \mu_{b}\), respectively, and common unknown variance \(\sigma^{2}\). (a) Show that the maximum likelihood estimators of \(c\) and \(d\) are, respectively, \(\hat{c}=\bar{X}_{. .}\) and $$ \hat{d}=\frac{\sum_{j=1}^{b}[j-(b-1) / 2]\left(\bar{X}_{. j}-\bar{X}_{. .}\right)}{\sum_{j=1}^{b}[j-(b+1) / 2]^{2}} $$ (b) Show that $$ \begin{aligned} \sum_{i=1}^{a} \sum_{j=1}^{b}\left(X_{i j}-\bar{X}_{. .}\right)^{2}=\sum_{i=1}^{a} \sum_{j=1}^{b}\left[X_{i j}-\bar{X}_{. .}-\hat{d}\left(j-\frac{b+1}{2}\right)\right]^{2} \\ &+\hat{d}^{2} \sum_{j=1}^{b} a\left(j-\frac{b+1}{2}\right)^{2} \end{aligned} $$ (c) Argue that the two terms in the right-hand member of Part (b), once divided by \(\sigma^{2}\), are independent random variables with \(\chi^{2}\) distributions provided that \(d=0\) (d) What \(F\) -statistic would be used to test the equality of the means, that is, \(H_{0}: d=0 ?\)

See all solutions

Recommended explanations on Math Textbooks

View all explanations

What do you think about this solution?

We value your feedback to improve our textbook solutions.

Study anywhere. Anytime. Across all devices.

Sign-up for free