Question

Improving SAT scores A national chain of SAT-preparation schools wants to know if using a smartphone app in addition to its regular program will help increase student scores more than using just the regular program. On average, the students in the regular program increase their scores by $128$ points during the $3-$month class. To investigate using the smartphone app, the prep schools have $5000$ students use the app along with the regular

program and measure their improvement. Then the schools will test the following hypotheses: $H_0:\mu =128$ versus $H_a:\mu >128$ , where$\mu$ is the true

mean improvement in the SAT score for students who attend these prep schools. After 3 months, the average improvement was $\overline{x}=130$ with a standard deviation of $s_x=65$ The standardized test statistic is $t=2.18$ with a $P-$value of $0.0148$ Explain why this result is statistically significant, but not practically important.

Short answer

The finding is statistically significant, but not practically meaningful because a two-point improvement on the SAT isn't that substantial.

Step-by-step solution

Given information

Hypotheses,

$$\begin{gathered} H_0:\mu =128 \\ {H}_a:\mu >128 \\ \overline{x}=130 \\ t=2.18 \\ {S}_x=65 \\ \alpha =0.0148 \end{gathered}$$

Calculation

The finding is statistically significant, but not practically meaningful because a two-point improvement on the SAT isn't that substantial.

There is good evidence of a difference when the null hypothesis can be rejected at the typical levels of $\alpha =0.05or\alpha =0.01$but the difference may be tiny. Small deviations from the null hypothesis will be noticeable even when huge samples are available.