Question

Let \(n\) independent trials of an experiment be such that \(x_{1}, x_{2}, \ldots, x_{k}\) are the respective numbers of times that the experiment ends in the mutually exclusive and exhaustive events \(C_{1}, C_{2}, \ldots, C_{k} .\) If \(p_{i}=P\left(C_{i}\right)\) is constant throughout the \(n\) trials, then the probability of that particular sequence of trials is \(L=p_{1}^{x_{1}} p_{2}^{x_{2}} \cdots p_{k}^{x_{k}}\). (a) Recalling that \(p_{1}+p_{2}+\cdots+p_{k}=1\), show that the likelihood ratio for testing \(H_{0}: p_{i}=p_{i 0}>0, i=1,2, \ldots, k\), against all alternatives is given by $$ \Lambda=\prod_{i=1}^{k}\left(\frac{\left(p_{i 0}\right)^{x_{i}}}{\left(x_{i} / n\right)^{x_{i}}}\right) $$ (b) Show that $$ -2 \log \Lambda=\sum_{i=1}^{k} \frac{x_{i}\left(x_{i}-n p_{i 0}\right)^{2}}{\left(n p_{i}^{\prime}\right)^{2}} $$ where \(p_{i}^{\prime}\) is between \(p_{i 0}\) and \(x_{i} / n\). Hint: Expand \(\log p_{i 0}\) in a Taylor's series with the remainder in the term involving \(\left(p_{i 0}-x_{i} / n\right)^{2}\). (c) For large \(n\), argue that \(x_{i} /\left(n p_{i}^{\prime}\right)^{2}\) is approximated by \(1 /\left(n p_{i 0}\right)\) and hence \(-2 \log \Lambda \approx \sum_{i=1}^{k} \frac{\left(x_{i}-n p_{i 0}\right)^{2}}{n p_{i 0}}\) when \(H_{0}\) is true. Theorem \(6.5 .1\) says that the right-hand member of this last equation defines a statistic that has an approximate chi-square distribution with \(k-1\) degrees of freedom. Note that dimension of \(\Omega-\) dimension of \(\omega=(k-1)-0=k-1\)

Short answer

The likelihood ratio, \(\Lambda\), for testing the hypothesis that the constant probability of an event in a series of trials equals a given value against all alternatives is \(\prod_{i=1}^{k}\left(\frac{p_{i0}^{x_{i}}}{(x_{i} / n)^{x_{i}}}\right)\). By taking logarithms, transforming using Taylor's series, and approximating for large \(n\), we find that -2 times the natural log of \(\Lambda\) becomes a chi-square distributed variable with \(k-1\) degrees of freedom when the null hypothesis is true.

Step-by-step solution

Setting Up the Likelihood Ratio

By definition, the likelihood ratio \(\Lambda\) under \(H_{0}\) is the ratio of the likelihood of observing this particular sequence of n trials if \(H_{0}\) is true versus if \(H_{0}\) is false. It is computed as: \( \Lambda = \frac{p_{1}^{x_{1}} p_{2}^{x_{2}} ... p_{k}^{x_{k}}}{(x_{1}/n)^{x_{1}} (x_{2}/n)^{x_{2}} ... (x_{k}/n)^{x_{k}}}\). Rewriting this in product notation, we get the desired formula \(\Lambda= \prod_{i=1}^{k} \left(\frac{(p_{i0})^{x_{i}}}{(x_{i} / n)^{x_{i}}}\right)\).

Apply Logarithm and Taylor's Series

Taking logarithms on both sides, we obtain: \(\log \Lambda = \sum_{i=1}^{k} x_{i} \log \left(\frac{p_{i0}}{x_{i}/n}\right)\). Using Taylor's series, we expand \(\log p_{i0}\) around \(x_{i}/n\) to get two terms and a remainder. Simplifying and multiplying both sides by -2 we get \( -2\log \Lambda = \sum_{i=1}^{k} \frac{x_{i}(x_{i}-n p_{i 0})^{2}}{(np'_{i})^{2}}\) where \(p'_{i}\) is between \(p_{i0}\) and \(x_{i}/n\).

Final Simplification and Argument

For larger \(n\), the ratio \(x_{i}/(np'_{i})^{2}\) is approximated by \(1/(np_{i0})\) as \(p'_{i}\) and \(p_{i0}\) become negligibly different. Hence, \( -2\log \Lambda \approx \sum_{i=1}^{k} \frac{(x_{i} - np_{i0})^2}{np_{i0}}\). It is then said that when \(H_{0}\) is true, the above expression defines a statistic that follows an approximate chi-square distribution with \(k-1\) degrees of freedom, justifying this with Theorem 6.5.1.

Key concepts

Chi-Square Distribution

The chi-square distribution is a crucial concept in both theoretical and applied statistics. It is a special case of the gamma distribution and is used predominantly in the field of hypothesis testing. The distribution itself is asymmetric and its shape is defined by the degrees of freedom, often denoted as df. In practical terms, the chi-square distribution helps us determine how likely an observed statistical result could have been due to chance.

The likelihood ratio test leverages the chi-square distribution to decide whether to reject a null hypothesis. Specifically, if we calculate a test statistic from our sample data and this statistic follows a chi-square distribution under the null hypothesis, we can compare this statistic against predefined thresholds called critical values to assess the strength of evidence against the null hypothesis. In the context of the problem at hand, for a large number of trials (n), the statistic -2 log has an approximate chi-square distribution with k - 1 degrees of freedom when the null hypothesis is true. This relationship is a pillar of assessing statistical significance in categorical data analysis.

Statistical Hypothesis Testing

Statistical hypothesis testing is a procedure used to make decisions based on data analysis, guided by probabilistic reasoning. It consists of comparing an observed sample result with what we expect to find in a population if a specific hypothesis were true. The primary aim is to ascertain the plausibility of this hypothesis given the data.

In the exercise, our goal is to apply a likelihood ratio test, which compares two competing hypotheses: the null hypothesis (), which we assume to be true, and an alternative hypothesis. If the likelihood ratio test points out that the data is highly unlikely under , it raises evidence against , potentially leading us to reject it in favor of the alternative. Remember, rejecting the null hypothesis does not prove the alternative true; it simply suggests that the data are not consistent with what we would expect under the null hypothesis, within a certain level of confidence.

Mathematical Statistics

Mathematical statistics is the branch of applied mathematics concerned with data collection, analysis, interpretation, and presentation. It is what allows statisticians to make sense of data by constructing mathematical models that reflect real-world phenomena.

The likelihood ratio test involved in our problem is a mathematical framework used to compare the fit of two models: one representing the null hypothesis and the other an alternative. The calculations involving likelihoods, logarithms, and their asymptotic distributions such as the chi-square are all embedded in the realm of mathematical statistics. These abstract concepts become tangible instruments for data-driven decision-making. In solving our exercise, we touched upon derivative principles like Taylor's series, which are part of the rich arsenal of tools that mathematical statistics provide to tackle such problems.

Probability Theory

Probability theory is the mathematical foundation that supports the entire structure of statistics and data analysis. It deals with the study of random events and is used to describe the likelihood of various outcomes. Its principles help to form the basis of statistical inference, which we rely on to draw conclusions about populations from samples.

In the context of the likelihood ratio test, probability theory gives us the language and tools to construct the concept of a probability distribution such as the chi-square distribution. It allows us to quantify the uncertainty involved in our hypothesis test, helping us to frame our conclusions not as certainties, but as statements with associated levels of confidence. Thus, when we say we 'reject the null hypothesis at the 5% significance level', we're essentially expressing that there's a 5% probability of rejecting a true null hypothesis, a concept deeply rooted in probability theory.