Question

Let \(X_{1}, X_{2}, \ldots, X_{n}\) and \(Y_{1}, Y_{2}, \ldots, Y_{m}\) be independent random samples from the two normal distributions \(N\left(0, \theta_{1}\right)\) and \(N\left(0, \theta_{2}\right)\). (a) Find the likelihood ratio \(\Lambda\) for testing the composite hypothesis \(H_{0}: \theta_{1}=\theta_{2}\) against the composite alternative \(H_{1}: \theta_{1} \neq \theta_{2}\). (b) This \(\Lambda\) is a function of what \(F\) -statistic that would actually be used in this test?

Short answer

The likelihood ratio \( \Lambda \) is a function of the sum of squared observations from two samples, and this is further related to the ratio of sample variances, i.e., \( F = s_{X}^{2}/s_{Y}^{2} \).

Step-by-step solution

Define the Hypothesis

First, let's define the null hypothesis \(H_{0}: \theta_{1}=\theta_{2}\) and the alternative hypothesis \(H_{1}: \theta_{1} \neq \theta_{2}\). These hypotheses are tests on the variances of the two different distributions. The null hypothesis \(H_{0}\) means that the variances of the two distributions are the same while the alternative hypothesis \( H_{1} \) means that the variances of the two distributions are not equal.

Construct the Likelihood Functions

Next, construct the likelihood functions for both \(H_{0}\) and \( H_{1} \). For \( H_{0} \), the likelihood function is \( L_{0} = \prod (f_{\theta}(x_{i})) \prod (f_{\theta}(y_{j})) \). For \( H_{1} \), the likelihood function is \( L_{1} = \prod (f_{\theta_1}(x_{i})) \prod (f_{\theta_2}(y_{j})) \). Here, \(f_{\theta}(x_{i})\) is the probability density function of \( X_{i} \) and \(f_{\theta}(y_{j})\) is the probability density function of \( Y_{j} \). Since the samples are drawn from normal distributions, the density functions are \(f_{\theta}(x_{i}) = (1/ \sqrt{2 \pi \theta}) \exp (-x_{i}^{2}/2\theta) \) and \(f_{\theta}(y_{j}) = (1/ \sqrt{2 \pi \theta}) \exp (-y_{j}^{2}/2\theta)\).

Calculate the Likelihood Ratio

Now, the likelihood ratio \( \Lambda \) for testing \( H_{0} \) against \( H_{1} \) can be calculated as \( \Lambda = L_{0}/L_{1} \). Plug in the likelihood functions from the previous step into the equation for \( \Lambda \). After calculating and simplifying, you find that \(\Lambda\) depends on the sum of squares of observations from two samples.

Identify the Corresponding F-statistic

The \( F \)-statistic that this \( \Lambda \) is a function of is the ratio of variances of the two samples. That is, \( F = s_{X}^{2}/s_{Y}^{2} \), where \( s_{X}^{2} \) and \( s_{Y}^{2} \) are the sample variances calculated from \( X_{1}, X_{2}, ..., X_{n} \) and \( Y_{1}, Y_{2}, ..., Y_{m} \) respectively.

Key concepts

Hypothesis Testing

Hypothesis testing is a foundational concept in statistics that allows researchers to make inferences about populations based on sample data. In its essence, hypothesis testing is about making decisions: it involves proposing a statement about a population parameter and then using sample data to decide whether there is enough evidence to reject that statement.

Let's simplify this with a real-world analogy. Suppose you declare that all swans are white, which is your null hypothesis (symbolized as H0). After observing many swans, if you encounter a black swan, you have evidence that contradicts your initial claim, leading you to reject the null hypothesis. In hypothesis testing, the null hypothesis is the default assumption that there's no effect or no difference, and we test whether sample data provide enough evidence to overturn this presumption.

For the exercise given, the null hypothesis (H0: θ1=θ2) asserts that the variances of two normal distributions are equal. The alternative hypothesis (H1: θ1 ≠ θ2) counters this, suggesting the variances are different. The likelihood ratio test—which compares the maximum likelihood estimations under the null and alternative hypotheses—is used to decide whether the observed data are strong enough to reject the null hypothesis in favor of the alternative.

Normal Distribution

The normal distribution, also known as the Gaussian distribution, is a continuous probability distribution that is symmetrical and has its mean, median, and mode at the same point. The hallmark of a normal distribution is its bell-shaped curve when graphed.

In the context of our statistical exercise, the random samples come from two separate normal distributions, each with a mean of zero but potentially different variances (θ1 and θ2). This shape is important because it influences the calculation of probabilities and thus how the data will be interpreted.

The properties of normal distributions are critical in hypothesis testing. They allow us to make probabilistic statements about how far sample statistics are likely to be from the population parameter under the null hypothesis. Due to the Central Limit Theorem, sample means tend to be normally distributed when the sample size is sufficiently large, making normal distribution a key player in hypothesis testing.

F-statistic

The F-statistic is a ratio used in statistical analysis to compare variances and is central to the analysis of variance (ANOVA) tests as well as the likelihood ratio test discussed in our exercise.

It essentially compares two sample variances to determine whether they are significantly different from each other. As such, the construction of an F-statistic is quite straightforward—it's the ratio of the variance of one sample to the variance of another sample (sX²/sY² in the context of the exercise).

If the null hypothesis is true and the underlying population variances are indeed equal, we would expect the F-statistic to be close to 1. Otherwise, a significant deviation from 1 suggests that the variances are different, aligning with the alternative hypothesis. To conclude if the difference is statistically significant, the F-statistic is compared to a critical value from the F-distribution, which is dependant on the sample sizes and chosen level of significance.

Sample Variance

Sample variance is a measure used to quantify the degree to which values in a sample differ from the sample mean—and by extension, from one another.

To calculate the sample variance (usually denoted s²), we subtract the mean from each data point, square the result, and then average those squared differences. Formally, for a sample of n observations, it's written as: \( s^2 = \frac{1}{n-1} \sum_{i=1}^{n}(x_i - \hat{x})^2 \), where \(x_i\) is an individual data point and \(\hat{x}\) is the sample mean. The \(n-1\) in the denominator is a correction factor for bias in estimating a population variance from a sample, often referred to as Bessel's correction.

In our exercise, the sample variances (sX² and sY²) serve as critical inputs into the F-statistic. Differences in these sample variances fuel the likelihood ratio test, helping to determine whether the data provide sufficient evidence to reject the null hypothesis of equal population variances.