Question

Let $X_1,X_2,\dots ,X_n$ be a random sample from the distribution $N({\theta }_1,{\theta }_2)$. Show that the likelihood ratio principle for testing $H_0:{\theta }_2={\theta }_2^{\prime }$ specified, and ${\theta }_1$ unspecified, against $H_1:{\theta }_2\ne {\theta }_2^{\prime },{\theta }_1$ unspecified, leads to a test that rejects when $\sum _1^n{(x_i-\overline{x})}^2\le c_1$ or $\sum _1^n{(x_i-\overline{x})}^2\ge c_2$, where \(c_{1}

Short answer

The likelihood ratio test principle leads to a test that rejects $H_0:{\theta }_2={\theta }_2^{\prime }$ when the sum $\sum _{i=1}^n(X_i-\overline{X}{)}^2$ is either quite small ($\le c_1$) or quite large ($\ge c_2$), where $c_1<c_2$, given $\alpha$, the size of the test.

Step-by-step solution

Develop Likelihood Function

The first step is to determine the likelihood of the given probability distribution. Given that $X_1,X_2,\dots ,X_n$ are random samples from a normal distribution $N({\theta }_1,{\theta }_2)$, the likelihood function $L({\theta }_1,{\theta }_2|X)$ is proportional to $\exp (-\frac{1}{2{\theta }_2^2}\sum _{i=1}^n{(X_i-{\theta }_1)}^2)$.

Constructing Test Statistic

The likelihood ratio test statistic is defined as $-2ln\lambda (X)$ where $\lambda (X)=\frac{supL(\theta |X;H_0)}{supL(\theta |X;H_1)}$. For $H_0:{\theta }_2={\theta }_2^{\prime }$, the MLE of ${\theta }_1$ is the sample mean under both hypotheses. Hence, $\lambda (X)$ simplifies to $\exp (-\frac{1}{2}[\frac{1}{{\theta }_2^{\prime 2}}-\frac{1}{{\widetilde{{\theta }_2}}^2}]\sum _{i=1}^n(X_i-\overline{X}{)}^2)$. Then, the likelihood ratio test statistic is $-2ln\lambda (X)=n[\frac{1}{{\theta }_2^{\prime 2}}-\frac{1}{{\widetilde{{\theta }_2}}^2}]\sum _{i=1}^n(X_i-\overline{X}{)}^2$.

Rejection Region

The rejection region can be formed by recognizing that we reject $H_0$ for large values of the test statistic. This will occur whenever $\sum _{i=1}^n(X_i-\overline{X}{)}^2$ is either quite small or quite large i.e. $\sum _{i=1}^n(X_i-\overline{X}{)}^2\le c_1$ or $\sum _{i=1}^n(X_i-\overline{X}{)}^2\ge c_2$, where $c_1<c_2$ must be selected so that the size of the test is $\alpha$.

Key concepts

Statistical Hypothesis Testing

Statistical hypothesis testing is a crucial procedure in statistics, used to determine whether there is sufficient evidence in a sample of data to conclude that a certain condition is true for the entire population. In simple terms, it's like a trial where we are testing an assumption (the null hypothesis, usually noted as $H_0$) against an alternative theory (the alternative hypothesis, $H_1$).

When performing hypothesis testing, we compute a test statistic, a numerical summary of data, that under certain assumptions follows a known probability distribution. If the value of this statistic is extreme, meaning it falls within the rejection region (which we'll discuss later), we reject $H_0$ in favor of $H_1$. The process entails an inherent risk of making errors – specifically, Type I error (rejecting a true null hypothesis) and Type II error (failing to reject a false null hypothesis). To mitigate these errors, we designate a significance level (usually denoted as $\alpha$), which is the probability of making a Type I error.

Normal Distribution

The normal distribution, also known as the Gaussian distribution, is a continuous probability distribution that is symmetric around the mean, showing that data near the mean are more frequent in occurrence than data far from the mean. In the context of hypothesis testing, the normal distribution's properties are particularly useful because many test statistics are assumed to follow it under the null hypothesis.

The normal distribution is defined by two parameters: the mean ${\theta }_1$ and the variance ${\theta }_2$. When using the normal distribution in hypothesis testing, as in the exercise provided, we often make inferences about these parameters based on the sample data. The assumption of normality allows us to use mathematical properties of the normal distribution to calculate probabilities and critical values related to our test statistic.

MLE (Maximum Likelihood Estimation)

Maximum Likelihood Estimation (MLE) is a method used to estimate the parameters of a statistical model. MLE selects the parameter values that make the observed data most probable. The beauty of MLE is that it often produces estimators with good statistical properties, such as consistency (as the sample size increases, the estimate converges to the true value) and efficiency (among all unbiased estimators, it has the smallest variance).

In the exercise, we utilize MLE to find the best-fitting parameters under both the null hypothesis $H_0$ and the alternative hypothesis $H_1$. Specifically, MLE is used to estimate ${\theta }_1$, the population mean, which is the same under both hypotheses and is simply the sample mean $\overline{X}$, due to the normal distribution of the data. However, the estimation of ${\theta }_2$, the variance, differs depending on whether we assume $H_0$ or $H_1$ is true.

Rejection Region

The rejection region is an essential concept in hypothesis testing that defines the range of values for which the null hypothesis can be rejected, and it is determined by the chosen significance level $\alpha$. It is shaped by the sampling distribution of the test statistic under the null hypothesis. In other words, it includes values of the test statistic that are very unlikely to occur if $H_0$ were true.

In the likelihood ratio test from our exercise, the rejection region is set by two critical values, $c_1$ and $c_2$, forming a two-tailed test. This means that if the test statistic falls below $c_1$ or above $c_2$, we have enough evidence to reject the null hypothesis. The values of $c_1$ and $c_2$ are chosen so that the probability of rejecting a true null hypothesis (Type I error) is equal to the significance level $\alpha$. The structure of the test and its placement of the rejection regions reflect the nature of the alternative hypothesis, which in this case does not specify whether the true variance is greater or less than the hypothesized value, warranting a two-tailed approach.