Question

A hypothesis test was correctly conducted and the experimenter failed to reject the null hypothesis. Which of the following must be true? I. The \(p\) -value was greater than \(\alpha\) II. A type I error did not occur. III. The power of the study was too small. (A) \(\quad\) I only (B) \(\quad \$ II only (C) \)\quad\( I and II only (D) \)\quad$ I and III only

Short answer

Condition I and II must be true. Choice (C) is correct.

Step-by-step solution

Understand hypothesis test terms

In a hypothesis test, the null hypothesis (H0) is the statement being tested and is presumed true unless evidence indicates otherwise. The significance level, \(\alpha\), is the probability threshold for rejecting H0.

Define conditions for failing to reject H0

Failing to reject H0 means we don't have sufficient evidence to disprove it. This corresponds to a situation where the \(p\)-value is greater than \(\alpha\). This is essential because the \(p\)-value indicates the probability of observing the data if H0 is true.

Discuss implications for type I error

Type I error occurs when we incorrectly reject H0. Since H0 was not rejected, Type I error did not occur.

Evaluate power of the study

The power of the study is the probability of correctly rejecting a false H0. Failing to reject H0 doesn't necessarily mean low power; it just means we don't have evidence against H0. Consider whether condition III is a guaranteed outcome.

Conclusion

Based on the reasoning, the conditions that must be true when failing to reject H0 are Condition I and Condition II. Therefore, the correct choice is (C).

Key concepts

null hypothesis

In hypothesis testing, the **null hypothesis** (denoted as H0) is a statement about a population parameter that we seek to test. This hypothesis represents a default or status quo position and is presumed true unless there is strong evidence to suggest otherwise. For instance, if we are testing whether a new drug is effective, the null hypothesis would state that the drug has no effect compared to a placebo.
When conducting a hypothesis test, we always assume H0 is true at the start. The objective is to gather sufficient data to determine whether there is enough evidence to reject H0 in favor of the alternative hypothesis (H1), which represents a contrary belief. Some common null hypotheses include:

• The mean of a population is equal to a specific value.
• The proportion of a population has a particular characteristic.
• There is no relationship between two variables.

Failing to reject the null hypothesis simply means that we do not have enough evidence to support H1, not that H0 is true beyond doubt.

significance level

The **significance level** (denoted as \(\function\alpha\)) in hypothesis testing is a threshold used to determine whether we reject the null hypothesis. It defines the probability of making a Type I error, which is incorrectly rejecting H0 when it is actually true. Common values for the significance level are 0.05, 0.01, and 0.10.
Here’s how it works:
During a test, we calculate a number called the **p-value**. This value measures the probability of obtaining the observed data, or something more extreme, if H0 is true. We then compare this p-value to \(\function\alpha\):

• If the p-value \(< \function\alpha\), we reject H0.
• If the p-value \(>= \function\alpha\), we fail to reject H0.

For example, if \(\function\alpha\) is set at 0.05 and our p-value comes out to be 0.03, we would reject H0 because 0.03 < 0.05.
The chosen significance level influences both the stringency of the test and the risk of Type I error. A smaller \(\function\alpha\) makes it harder to reject H0, reducing the likelihood of a Type I error but potentially increasing the likelihood of a Type II error (failing to reject a false H0).

Type I error

**Type I error** occurs when the null hypothesis (H0) is wrongly rejected in favor of the alternative hypothesis (H1). In other words, we conclude there is an effect or difference when in reality, there is none. This type of error is considered false positive.
Type I error is directly related to the significance level (\(\function\alpha\)). If we set \(\function\alpha\) at 0.05, we accept that there is a 5% risk of committing a Type I error. Selecting a lower \(\function\alpha\), like 0.01, decreases this risk but makes it harder to detect true effects when they exist.
To put it in context, if we are testing a new medication, a Type I error would mean approving the drug as effective when it is not. Controlling Type I error is crucial in ensuring that we only make claims supported by sufficient evidence.

• Setting \(\function\alpha\) too high can lead to more Type I errors.
• Setting \(\function\alpha\) too low can make it difficult to detect true effects, thus committing Type II errors (false negatives).

Balancing Type I and Type II errors is essential in hypothesis testing to make reliable and accurate inferences.