Question

Patients with nonchronic low back pain were randomly assigned to either a treatment group that received health coaching via telephone plus physiotherapy care or a control group that received only the physiotherapy care. One variable measured was improvement, over 4 weeks, in "modified Oswestry Disability Index," which is scored as a percentage, ranging from 0 to 100 , with high scores representing high levels of disability. Summary statistics are shown in the table. \({ }^{54}\) $$ \begin{array}{|lcc|} \hline & \text { Experimental } & \text { Control } \\ \hline \text { Mean } & 16.8 & 11.9 \\ \text { SD } & 18.7 & 11.6 \\ n & 12 & 14 \\ \hline \end{array} $$ (a) Calculate the sample effect size from these data. (b) A \(95 \%\) confidence interval for the difference in population mean Oswestry Disability Index for the two groups \(\left(\mu_{\text {Experimental }}-\mu_{\text {Control }}\right)\) is (-8.2,18.0) percentage points. The researchers were hoping to find a difference of at least 5 percentage points, a difference that they thought would be clinically important. Based on the confidence interval, would they be ill-advised to conduct a larger study in the hope of finding convincing evidence of a 5 percentage point difference? (c) How would your answer to (b) change if the confidence interval was (-2.4,3.2) percentage points.

Short answer

The sample effect size calculated from the data is Cohen's d. The initial 95% confidence interval (-8.2, 18.0) includes the 5 percentage point difference, suggesting some possibility of this effect, so a larger study may be reasonable. However, a revised confidence interval (-2.4, 3.2) does not include the 5 percentage point difference, implying that conducting a larger study to find a clinically important difference of 5 points would be ill-advised.

Step-by-step solution

Calculate the Sample Effect Size

To calculate the effect size, known as Cohen's d, we use the difference between the two means and the pooled standard deviation (SD). The formula for Cohen's d is: Cohen's d = (Mean_1 - Mean_2) / SD_pooled where SD_pooled = sqrt[((n1-1)SD1^2 + (n2-1)SD2^2) / (n1 + n2 - 2)].First, we need to calculate SD_pooled with the given data.

Calculate the Pooled Standard Deviation

Using the formula: SD_pooled = sqrt[((n1-1)SD1^2 + (n2-1)SD2^2) / (n1 + n2 - 2)], where n1 = 12, n2 = 14, SD1 = 18.7, and SD2 = 11.6, solve for SD_pooled.

Calculate Cohen's d

Now that we have SD_pooled, use the formula: Cohen's d = (Mean_1 - Mean_2) / SD_pooled, where Mean_1 = 16.8 and Mean_2 = 11.9, to calculate the effect size.

Interpret the 95% Confidence Interval for the Mean Difference

Given the 95% confidence interval (-8.2, 18.0) for the difference of means, if this interval does not include the value of interest for the difference in means (5 percentage points), then the researchers have some evidence that the difference could be at least 5 percentage points.

Assess the Feasibility of a Larger Study

Since the confidence interval (-8.2, 18.0) includes 5, it suggests there is no conclusive evidence to support a difference of at least 5 percentage points. However, because the interval also includes values greater than 5, the researchers may still find it reasonable to conduct a larger study.

Reassess Based on the New Confidence Interval

With a confidence interval of (-2.4, 3.2), which does not include the 5 percentage point difference the researchers are looking for, this indicates that even if a difference exists, it may not be clinically important or as large as hoped for. Therefore, based on this interval, it would not be advisable to conduct a larger study in the hope of finding convincing evidence of a 5 percentage point difference.

Key concepts

Cohen's d Calculation

Understanding the magnitude of an experimental treatment's effect is crucial in research, and this is where the Cohen's d calculation comes into play. Cohen's d is a measure of effect size that helps to quantify the difference between two group means in relation to their standard deviation, essentially providing insight into the practical significance of treatment outcomes.

The formula for calculating Cohen's d is relatively straightforward: \[ Cohen's d = \frac{\text{Mean}_1 - \text{Mean}_2}{SD_{pooled}} \], where \( SD_{pooled} \) is a combined standard deviation that accounts for both groups’ variability. To calculate \( SD_{pooled} \) , the formula \[ SD_{pooled} = \sqrt{\frac{((n1-1)SD1^2 + (n2-1)SD2^2)}{(n1 + n2 - 2)}} \] is used, incorporating sample sizes and standard deviations from both groups.

For educators and students alike, it's important to interpret Cohen's d in the context of the study. Generally, a Cohen's d of 0.2 is considered a 'small' effect size, 0.5 is 'medium', and 0.8 or above is 'large'. However, what constitutes a 'small' or 'large' effect can vary by field and context.

95% Confidence Interval

A 95% confidence interval is a fundamental concept in statistics that provides a range of values within which we can be 95% confident the true population parameter lies. It is calculated from sample data and provides a measure of the precision of our estimate. For example, the 95% confidence interval for the difference in population mean Oswestry Disability Index scores between experimental and control groups gives us a range where the true mean difference might fall.

In context of our exercise, a 95% confidence interval that includes zero suggests that there is no significant difference between the two groups. If the interval includes the value of clinical importance (in this case, a 5-point difference), it indicates that such a difference is plausible but not statistically confirmed in this sample. On the other hand, if the interval does not include 5, we would infer that a difference of 5 points is unlikely to arise by chance alone, possibly indicating its absence in the true population. Thus, the confidence interval can guide researchers in making decisions about whether further study is warranted.

Oswestry Disability Index

The Oswestry Disability Index (ODI) is a specific instrument used in the field of healthcare to assess a patient's permanent functional disability. This index is particularly used for measuring the progress and severity of back pain, which is the variable of interest in our exercise concerning nonchronic low back pain treatment effects.

The ODI is a scoring system that ranges from 0 to 100, where higher scores indicate greater disability. It is comprised of several questions related to everyday activities and the patient's ability to carry them out without pain or discomfort. In clinical studies and practice, a change in ODI score is often used to determine the effectiveness of treatments, such as physiotherapy or health coaching, and can be a prime target for measuring outcomes. When evaluating treatment effects, it's vital to consider both the statistical significance (as measured by tests and CIs) and the clinical relevance, which is often represented by the effect size on practical scales like the ODI.