Question

Question: Statistical Significance versus Practical Significance. When testing a new treatment, what is the difference between statistical significance and practical significance? Can a treatment have statistical significance but not practical significance?

Short answer

The distinction between statistical and practical significance is as follows:

• Statistical significance denotes that the test result is unlikely to arise.

• Practical significance shows whether or not the treatment suggests enough difference between the sample and parameter (or hypothetical value).

Yes, a treatment that is not practically significant can have statistical significance.

Step-by-step solution

Compare the statistical significance and practical significance of a new treatment.

Statistical significances are results of statistical tests, termed hypothesis tests. They conclude results based on sample statistics like sample size and other estimates. A statistically significant result implies that the new treatment shows a change or difference from the null assumption.

The result is obtained when a rare sample value is acquired, assuming the null assumption is correct.

Practical significance, on the other hand, is measured using the effect sizes of the hypothesis tests. It indicates whether the statistical conclusion holds some practical importance, whether meaningful for common use or not.

Thus, statistical significance decides if the sample results apply to the entire population, while practical significance decides if the results are practical enough to be mapped onto the population for common use.

Explain if a treatment can have statistical significance but no practical significance.

Yes, a new treatment being tested may have statistical significance and yet no practical significance.

This situation can occur in the following scenarios:

• The sample size is too large, which leads to statistical significance, but it is of no practical use.

• The samples are collected in a way that the dispersion of observation is too small, which can also lead to a statistically significant result that may not be used for a common purpose in the population