Question

Let \(X_{1}, X_{2}, \ldots, X_{n}\) be iid \(N\left(\theta_{1}, \theta_{2}\right) .\) Show that the likelihood ratio principle for testing \(H_{0}: \theta_{2}=\theta_{2}^{\prime}\) specified, and \(\theta_{1}\) unspecified, against \(H_{1}: \theta_{2} \neq \theta_{2}^{\prime}, \theta_{1}\) unspecified, leads to a test that rejects when \(\sum_{1}^{n}\left(x_{i}-\bar{x}\right)^{2} \leq c_{1}\) or \(\sum_{1}^{n}\left(x_{i}-\bar{x}\right)^{2} \geq c_{2}\) where \(c_{1}

Short answer

The likelihood ratio principle for testing \(H_{0}:\theta_{2}=\theta_{2}^{'}\), \(\theta_{1}\) unspecified against \(H_{1}: \theta_{2} \neq \theta_{2}^{'}\), \(\theta_{1}\) unspecified, can lead to a test that rejects when \(\sum_{1}^{n}\left(x_{i}-\bar{x}\right)^{2} \leq c_{1}\) or \(\sum_{1}^{n}\left(x_{i}-\bar{x}\right)^{2} \geq c_{2}\). This is due to the nature of the likelihood ratio test and the fact that the sum of squares of residuals follows a chi-square distribution under the null hypothesis. The critical values \(c_{1}\) and \(c_{2}\) should be set based on the quantiles of this distribution to maintain a certain Type I Error rate.

Step-by-step solution

Understand and set up the problem

The null hypothesis \(H_{0}\) is that the variance \(\theta_{2} = \theta_{2}^{'}, \theta_{1}\) is unspecified and the alternative hypothesis \(H_{1}\) is \(\theta_{2} \neq \theta_{2}^{'}, \theta_{1}\) is unspecified. The likelihood ratio principle for testing these hypotheses states that the ratio of the likelihood of the observed data under the null hypothesis to the likelihood under the alternate hypothesis is used to decide whether to reject the null hypothesis.

Define the likelihood function

The likelihood function for a normal distribution is \(L(\theta_{1}, \theta_{2}) = \frac{1}{(\sqrt{2\pi\theta_{2}})^{n}} e^{-\frac{1}{2\theta_{2}} \sum_{i=1}^{n} (x_{i}-\theta_{1})^{2}}\).\nNext, we need to calculate the log-likelihood function, which simplifies calculations considerably. The log of the likelihood function is given by \(log(L(\theta_{1}, \theta_{2})) = -\frac{n}{2} log(2\pi) - n log(\sqrt{\theta_{2}}) - \frac{1}{2\theta_{2}} \sum_{i=1}^{n} (x_{i}-\theta_{1})^{2}\)

The likelihood ratio test statistic (LRT)

The LRT can be expressed as \(-2log(\lambda)\), where \(\lambda = L(\theta_{1}, \theta_{2}^{'} | \mathbb{X}) / L(\theta_{1}, \theta_{2} | \mathbb{X})\). Here, \(\theta_{2}^{'}\) is the hypothesised value under the null, and \( \mathbb{X}\) represents the observed data. After simplifying, this leads: \(-2log(\lambda) = -2[nlog( \sqrt{\theta_{2}}) - nlog( \sqrt{\theta_{2}^{'}}) - \frac{1}{2\theta_{2}} \sum_{i=1}^{n} (x_{i}-\theta_{1})^{2} + \frac{1}{2\theta_{2}^{'}} \sum_{i=1}^{n} (x_{i}-\theta_{1})^{2}] = 2nlog(\frac{\sqrt{\theta_{2}}}{\sqrt{\theta_{2}^{'}}}) - \frac{\sum_{i=1}^{n} (x_{i}-\theta_{1})^{2}}{\theta_{2}} + \frac{\sum_{i=1}^{n} (x_{i}-\theta_{1})^{2}}{\theta_{2}^{'}}\)

Expressing the test statistic in terms of the given condition

Notice that \(\sum_{i=1}^{n} (x_{i}-\theta_{1})^{2}\) is the estimate of the variance and can be calculated using the observed data, denoted as \(\sum_{1}^{n}\left(x_{i}-\bar{x}\right)^{2}\). Inserting the estimated value of the variance into the place of our \(\sum_{i=1}^{n} (x_{i}-\theta_{1})^{2}\) in our LRT expression, we wind up at -2log(\(\lambda)\) = \(2nlog(\sqrt{\theta_{2}/\theta_{2}^{'}} + \frac{\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}}{\theta_{2}} - \frac{\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}}{\theta_{2}^{'}}\). Our rejection criterion for \(H_{0}\) can now be expressed as \(c_{1} <= \sum_{i=1}^{n}(x_{i}-\bar{x})^{2} <= c_{2}\).

Selection of \(c_{1}\) and \(c_{2}\)

The selection of \(c_{1}\) and \(c_{2}\) should be done such that the probability of Type I error (rejecting \(H_{0}\) when it is true) is controlled at a pre-specified level \(\alpha\). For this exercise, we were asked to demonstrate that such \(c_{1}\) and \(c_{2}\) exist, not to find them. To prove their existence, consider the fact that \(\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}\) follows a chi-square distribution with \(n\) degrees of freedom under the null hypothesis, and we can select the critical values \(c_{1}\) and \(c_{2}\) as the \(\alpha/2\) and \(1-\alpha/2\) quantiles of this distribution, thus controlling the Type I error rate at \(\alpha\).