Question

Using the notation of this section, assume that the means satisfy the condition that \(\mu=\mu_{1}+(b-1) d=\mu_{2}-d=\mu_{3}-d=\cdots=\mu_{b}-d .\) That is, the last \(b-1\) means are equal but differ from the first mean \(\mu_{1}\), provided that \(d \neq 0\). Let independent random samples of size \(a\) be taken from the \(b\) normal distributions with common unknown variance \(\sigma^{2}\). (a) Show that the maximum likelihood estimators of \(\mu\) and \(d\) are \(\hat{\mu}=\bar{X}_{. .}\) and $$ \hat{d}=\frac{\sum_{j=2}^{b} \bar{X}_{. j} /(b-1)-\bar{X}_{.1}}{b} $$ (b) Using Exercise \(9.2 .4\), find \(Q_{6}\) and \(Q_{7}=c \hat{d}^{2}\) so that, when \(d=0, Q_{7} / \sigma^{2}\) is \(\chi^{2}(1)\) and $$ \sum_{i=1}^{a} \sum_{j=1}^{b}\left(X_{i j}-\bar{X}_{. .}\right)^{2}=Q_{3}+Q_{6}+Q_{7} $$ (c) Argue that the three terms in the right-hand member of part (b), once divided by \(\sigma^{2}\), are independent random variables with chi-square distributions, provided that \(d=0\). (d) The ratio \(Q_{7} /\left(Q_{3}+Q_{6}\right)\) times what constant has an \(F\) -distribution, provided that \(d=0\) ? Note that this \(F\) is really the square of the two-sample \(T\) used to test the equality of the mean of the first distribution and the common mean of the other distributions, in which the last \(b-1\) samples are combined into one.

Short answer

The maximum likelihood estimators of \( \mu \) and \( d \) are found to be \( \hat{\mu}=\bar{X}_{. .} \) and \( \hat{d}=\frac{\sum_{j=2}^{b} \bar{X}_{. j} /(b-1)-\bar{X}_{.1}}{b} \). \( Q_6 \) and \( Q_7 \) constituting total sum of squares are derived, leading to chi-square distributions of the three terms in the right side of equation for condition \( d=0 \). The ratio \( Q_{7} /\left(Q_{3}+Q_{6}\right) \) times a constant follows an \( F \)-distribution.

Step-by-step solution

Find Maximum Likelihood Estimators

The maximum likelihood estimators of \( \mu \) and \( d \) are given as:\( \hat{\mu}=\bar{X}_{. .} \), which indicates that the estimator \( \hat{\mu} \) for \( \mu \) is the grand mean (mean of all the distributions).\( \hat{d}=\frac{\sum_{j=2}^{b} \bar{X}_{. j} /(b-1)-\bar{X}_{.1}}{b} \), means that the estimator \( \hat{d} \) for \( d \) is the result of subtracting the mean of first distribution \( \bar{X}_{.1} \) from the mean of means for remaining distributions divided by b.

Find \( Q_{6} \) and \( Q_{7} \)

To find \( Q_6 \) and \( Q_7 \), Exercise \( 9.2.4 \) is used, where\( Q_6 \) and \( Q_7 \) have been derived earlier. Now, all the three terms on the right hand of equation:\[ \sum_{i=1}^{a} \sum_{j=1}^{b}(X_{i j}-\bar{X}_{..})^{2}=Q_{3}+Q_{6}+Q_{7}\] equal the total sum of squares, which is a well-known result in statistics.

Argue Chi-Square Distributions

If the condition \( d=0 \) is met, the three terms in the right-hand member of equation can be transformed to be independent random variables with chi-square distributions. This can be argued as, if \( d=0 \), it means \( \mu_1 = \mu_2 = \mu_3 =...= \mu_b \), hence the deviations of \( X_{ij} \) from their expectation \(\mu \) are independent and each divided by \( \sigma^2 \) follows chi-square distribution.

Identify F-Distribution

The ratio \( Q_{7} /\left(Q_{3}+Q_{6}\right) \), when multiplied by an appropriate constant, has an \( F \)-distribution, provided that \( d=0 \). The \( F \) here is actually the square of a two-sample \( T \) statistic, testing whether the mean of the first distribution and the common mean of the other distributions are equal. This implies that \( b-1 \) samples are combined into one.

Key concepts

Chi-Square Distribution

The chi-square distribution is a fundamental concept in statistics often used to measure how expectations compare to actual observed data. It comprises a family of distributions that take on only positive values and are distinctly skewed to the right, particularly useful when dealing with positive random variables such as variance.

This distribution is frequently employed in hypothesis testing and confidence interval estimation for scenarios involving variance and standard deviation of a normal distribution. When we assume that each sample comes from a normal distribution with the same unknown variance, \( \sigma^2 \), any sample variance calculated mimics a scaled chi-square distribution.

In the context of the textbook exercise, when the null hypothesis holds (that is, \( d=0 \) indicating all population means are equal), the third term calculated in part (b), divided by the variance, follows a chi-square distribution with one degree of freedom, \( \chi^2(1) \). This is crucial because it allows us to create a test statistic for hypothesis testing.

F-Distribution

The F-distribution is another critical probability distribution in statistics, especially in the context of variance analysis and related hypothesis testing, such as the ANOVA test. It is the distribution of the ratio of two scaled chi-square distributions, each divided by their respective degrees of freedom.

Understanding the F-distribution is vital because it allows statisticians to compare variances and determine if they come from populations with the same variance, which is essential in many practical experiments.

In the given exercise, part (d) is essentially asking for the scale factor that turns the ratio \( Q_{7} /(Q_{3}+Q_{6}) \) into an F-distributed variable when \( d=0 \). The F-statistic constructed in this manner can be used to test the equality of two variances—in this case, to verify if the mean of the first distribution is the same as the common mean of the remaining distributions.

Statistical Hypothesis Testing

Statistical hypothesis testing is a method used to determine whether there is enough evidence in a sample of data to infer that a certain condition is true for the entire population. A hypothesis test usually starts with a null hypothesis (\( H_0 \) - representing a default position or status quo) and an alternative hypothesis (\( H_A \), what the test is trying to establish).

In such tests, deciding between using a chi-square or an F-distribution largely depends on the nature of the hypothesis being tested. As per the exercise, the use of chi-square distribution in part (b) or F-distribution in part (d) revolves around the assumptions of common variance and normally distributed populations. Therefore, demonstrating the independence and distribution of \( Q_{3}, Q_{6}, \) and \( Q_{7} \) under the assumption that \( d=0 \) is essential for creating an appropriate test statistic.

Ultimately, the exercise demonstrates how maximum likelihood estimators can be used to derive test statistics that follow a specific distribution, either chi-square or F-distribution, for statistical hypothesis testing.