Question

The degree of success at mastering a skill often depends on the method used to learn the skill. The article "Effects of Occluded Vision and Imagery on Putting Golf Balls" (Perceptual and Motor Skills [1995]: \(179-186\) ) reported on a study involving the following four learning methods: (1) visual contact and imagery, (2) nonvisual contact and imagery, (3) visual contact, and (4) control. There were 20 subjects randomly assigned to each method. The following summary information on putting performance score was reported: $$\begin{array}{cccccc} \text { Method } & 1 & 2 & 3 & 4 & \\ \hline \bar{x} & 16.30 & 15.25 & 12.05 & 9.30 & \overline{\bar{x}}=13.23 \\ s & 2.03 & 3.23 & 2.91 & 2.85 & \end{array}$$ a. Is there sufficient evidence to conclude the mean putting performance score is not the same for the four methods? b. Calculate the \(95 \%\) T-K intervals, and then use the underscoring procedure described in this section to identify significant differences among the learning methods.

Short answer

We firstly perform an ANOVA test. If we reject the null hypothesis, then we continue with Tukey's method to make multiple comparisons and find which pairs of groups are significantly different from each other.

Step-by-step solution

Setup Hypotheses and Collect Data

Set up the null hypothesis \(H_0\): the means of the four groups are equal, \(\mu_1 = \mu_2 = \mu_3 = \mu_4\), and the alternative hypothesis \(H_1\): at least one mean is different from the others. From the problem, we gather the sample means \(\bar{x}_i\) and sample standard deviations \(s_i\) for each group, along with the number of samples in each group (n = 20).

Perform ANOVA Test

Perform an Analysis of Variance (ANOVA) to test the null hypothesis. To do this, first we calculate the total mean \(\overline{\bar{x}}\), the within-group variance \(S^2_W\) and between-group variance \(S^2_B\). Then, compute the F statistic as the ratio of between-group variance to within-group variance, \(F = S^2_B / S^2_W\). Finally, we compare the calculated value of the F statistic with the critical value from the F distribution table for \(\alpha = 0.05\), three degrees of freedom in the numerator (df1 = 4-1), and 76 degrees of freedom in the denominator (df2 = 80-4). If our calculated F statistic is greater than the critical value, we reject the null hypothesis, which means it's likely that at least one mean is different from the others.

Multiple Comparisons with Tukey's Method

If we have rejected the null hypothesis in step 2, we can continue with multiple comparisons among the means. Use Tukey's method to calculate the 95% Tukey-Kramer (T-K) intervals. These intervals give a range of values that would contain the true difference in the population means between any two groups with 95% confidence. If zero is not contained within the computed interval for a pair of groups, it means their population means are statistically significantly different. This step will help identify which groups are significantly different from each other.

Key concepts

Null Hypothesis in Statistics

The null hypothesis is a foundation of statistical testing, representing the default assumption that there is no effect or no difference between groups. For example, when we are comparing mean scores of different learning methods in an experiment, we start by assuming that the mean scores are equal across all groups—that is, any observed differences are due to random variation.

In essence, the null hypothesis (\(H_0\)) for the putting performance score experiment is formulated as \(\mu_1 = \mu_2 = \mu_3 = \mu_4\), where the mean (\(\mu\)) represents the putting performance for each method (1 through 4). It's an essential step because it creates a benchmark for testing whether the observed data provide enough evidence to suggest a real difference exists.

F-Statistic

The F-statistic plays a crucial role in the Analysis of Variance (ANOVA) test. It's a ratio which compares the amount of systematic variance between different groups to the amount of unsystematic variance within the groups.

Mathematically, it is expressed as \( F = S^2_B / S^2_W \), where \( S^2_B \) is the variance between the group means and \( S^2_W \) is the variance within the groups. A larger F-statistic indicates a greater probability that the observed variation between group means is not just due to chance. If the calculated F-statistic exceeds the critical value obtained from an F distribution table, the null hypothesis is rejected.

Tukey's Honest Significance Test

Tukey's Honest Significance Test, also known as Tukey's HSD (Honestly Significant Difference) Test, is a post-hoc analysis conducted after an ANOVA when we find a statistically significant result. It allows for multiple comparisons without increasing the possibility of a Type I error, which occurs when falsely claiming a difference when none exists.

This test compares all possible pairs of means and determines which means are significantly different from each other. It provides ranges of values (Tukey-Kramer intervals) for the differences between group means, and if zero is not included within these intervals, it suggests a statistically significant difference between those particular groups.

Multiple Comparisons in Statistics

Multiple comparisons occur when we test several statistical comparisons at once. A key issue here is the risk of Type I errors; as more comparisons are made, the chance of incorrectly rejecting a true null hypothesis increases. It’s akin to looking for a signal in increasing amounts of noise.

Statisticians use various methods to control this risk, such as Tukey’s method, which adjusts the confidence intervals for the pair-wise differences between means so that the overarching 'family' error rate remains at a chosen level. This ensures that significant results are not just due to random chance but signal a true difference between the tested groups.

Analysis of Variance

Analysis of Variance (ANOVA) is a statistical test used to compare the means of three or more independent groups to see if at least one group mean is statistically different from the others. It separates total variance observed in the data into two components: variance between groups and variance within groups.

ANOVA is based on the F-statistic and assumes that the data is normally distributed and that the variances of the groups are equal (homogeneity of variance). If the ANOVA yields a significant result, further analysis with post-hoc tests, such as Tukey’s HSD, will typically follow to pinpoint exactly where the differences lie.

Statistical Significance

Statistical significance is a determination that the result observed in a dataset is not likely due to chance alone. It is often determined by a p-value, which if below a predetermined threshold (usually 0.05), suggests that the observed differences or associations are 'significant'.

In the context of ANOVA, if the F-statistic leads to a p-value less than 0.05, we say the result is statistically significant. This would imply that the differences in group means are large enough that we can confidently reject the null hypothesis, going beyond the natural fluctuation expected with random samples.