Question

Do iPads Help Kindergartners Learn: A Subtest The Auburn, Maine, school district conducted an early literacy experiment in the fall of 2011 . In September, half of the kindergarten classes were randomly assigned iPads (the intervention group) while the other half of the classes got them in December (the control group.) Kids were tested in September and December and the study measures the average difference in score gains between the control and intervention group. \(^{41}\) The experimenters tested whether the mean score for the intervention group was higher on the HRSIW subtest (Hearing and Recording Sounds in Words) than the mean score for the control group. (a) State the null and alternative hypotheses of the test and define any relevant parameters. (b) The p-value for the test is 0.02 . State the conclusion of the test in context. Are the results statistically significant at the \(5 \%\) level? (c) The effect size was about two points, which means the mean score for the intervention group was approximately two points higher than the mean score for the control group on this subtest. A school board member argues, "While these results might be statistically significant, they may not be practically significant." What does she mean by this in this context?

Short answer

a) Null hypothesis (H0): The mean score for the intervention group is same as that for the control group. Alternative hypothesis (H1): The mean score for the intervention group is higher than that for the control group. b) With a p-value of 0.02, the results are statistically significant at the 5% level. This means we reject the null hypothesis in favor of the alternative hypothesis. c) While the test results are statistically significant, the practical significance is in question here. A two points difference, while statistically significant, may not bring a substantial difference in real world application.

Step-by-step solution

Null and Alternative Hypotheses

The null hypothesis (H0) typically proposes that there is no effect of the intervention. In this context, it means there is no difference in the mean scores between the intervention group (those with iPads) and the control group (those without iPads initially). So, H0: \(\mu_{int} = \mu_{ctrl}\), where \(\mu_{int}\) and \(\mu_{ctrl}\) are the population mean scores for the intervention and control groups respectively.The alternative hypothesis (H1) proposes the contrary, specifically that there is an effect. In this case, it would be that the intervention group's mean is higher than the control group's. Therefore, H1: \(\mu_{int} > \mu_{ctrl}\)

Interpreting the P-value

The p-value is a measure of how extreme the data are. In this case, a p-value of 0.02 suggests that the likelihood of seeing the observed data (or more extreme), given that the null hypothesis is true, is just 0.02, or 2%.

Statistical Significance

A result is typically considered statistically significant if the p-value is less than a predetermined threshold (commonly 5%). Since the given p-value of 0.02 is less than 0.05, the result is statistically significant at the 5% level. This means that there is significant evidence to reject the null hypothesis in favor of the alternative hypothesis - suggesting that the iPads had a significant effect on the mean score of kindergarteners.

Statistical Significance vs. Practical Significance

Practical significance refers to the magnitude of the difference, whether it's large enough to be of value in a practical sense. The effect size here is about two points. This implies that while there may be a statistically significant improvement, whether this improvement is practically significant (i.e., large enough to bring substantial difference or value in real world application) will depend on the conventional standards, professional judgement or the context of usage. The school board member is pointing out that while statistically there is an improvement, it may not bring substantial difference or value in practical usage or application.