Question

Let \(Y_{1}, Y_{2}, \ldots, Y_{n}\) be \(n\) independent normal variables with common unknown variance \(\sigma^{2}\). Let \(Y_{i}\) have mean \(\beta x_{i}, i=1,2, \ldots, n\), where \(x_{1}, x_{2}, \ldots, x_{n}\) are known but not all the same and \(\beta\) is an unknown constant. Find the likelihood ratio test for \(H_{0}: \beta=0\) against all alternatives. Show that this likelihood ratio test can be based on a statistic that has a well-known distribution.

Short answer

The likelihood ratio test for the hypothesis can be based on a test statistic that follows a T-distribution with n-2 degrees of freedom.

Step-by-step solution

Setting up the likelihoods

The likelihood for our model given the null hypothesis \(H_{0}: \beta=0\) is given by \(L_0 = \prod _{i=1}^{n}\frac{1}{\sqrt{2\pi\sigma^{2}}} e^{\frac{-(y_{i}-0)^2}{2\sigma^{2}}}\). The likelihood for our model given the alternative hypothesis \(H_{1}: \beta\neq0\) is \(L_1 = \prod_{i=1}^{n}\frac{1}{\sqrt{2\pi\sigma^{2}}} e^{\frac{-(y_{i}-\beta x_{i})^2}{2\sigma^{2}}}\).

Calculating the likelihood ratio

The likelihood ratio is the ratio \(LR = \frac{L_1}{L_0}\). Take the natural logarithm of both sides then simplify. The result is \(ln(LR) = \frac{1}{2\sigma^{2}} \sum_{i=1}^{n} (y_{i}- \beta x_{i})^2 - \sum_{i=1}^{n} y_{i}^2\). The maximum likelihood estimator\(\hat{\beta}\) under the alternative hypothesis \(H_{1}: \beta=0\) maximizes \(LR\), so we take the derivative of \(ln(LR)\) with respect to \(\beta\), set it equal to zero, and solve for \(\beta\). Then substitute \(\hat{\beta}\) back into \(ln(LR)\).

Re-expressing the likelihood ratio in terms of a known test statistic

Now that we have \(LR\), we can express it in terms of another statistic. This will be easier to work with and understand since the distribution of this new statistic will be well-known. When choosing the new statistic, it should ideally be related to \(\beta\) and \(\sigma\), as these are the parameters we are interested in for the test. Also, consider the F-distribution, T-distribution, or Chi-squared distribution, particularly if we are looking at ratios of variances or sums of squares.

Identifying the Distribution

By re-expressing and simplifying \(ln(LR)\), we can see it follows a T-distribution, with \(T = \frac{\hat{\beta}}{SE(\hat{\beta})}\) where \(SE(\hat{\beta})\) is the standard error of \(\hat{\beta}\). We can view the fraction as the T-statistic, where \(T\) is normally distributed with n-2 degrees of freedom. The statistic now resembles a common hypothesis test for \(\beta\), such as the ones we would use in regression analysis.