Question

The article "Utilizing Feedback and Goal Setting to Increase Performance Skills of Managers" (Academy of Management Journal \([1979]: 516-526\) ) reported the results of an experiment to compare three different interviewing techniques for employee evaluations. One method allowed the employee being evaluated to discuss previous evaluations, the second involved setting goals for the employee, and the third did not allow either feedback or goal setting. After the interviews were concluded, the evaluated employee was asked to indicate how satisfied he or she was with the interview. (A numerical scale was used to quantify level of satisfaction.) The authors used ANOVA to compare the three interview techniques. An \(F\) statistic value of \(4.12\) was reported. a. Suppose that a total of 33 subjects were used, with each technique applied to 11 of them. Use this information to conduct a level \(.05\) test of the null hypothesis of no difference in mean satisfaction level for the three interview techniques. b. The actual number of subjects on which each technique was used was \(45 .\) After studying the \(F\) table, explain why the conclusion in Part (a) still holds.

Short answer

a. The null hypothesis of no difference in mean satisfaction level for the three interviewing techniques is rejected, as the F-value (\(4.12\)) is greater than the critical value (\(3.32\)) at a \(.05\) significance level. \n\n b. Our conclusion from part (a) still holds even if the number of subjects increases to 45, because our F-value (\(4.12\)) would still be significant at a \(.05\) significance level. There is a statistically significant difference in mean satisfaction levels between the three interview techniques.

Step-by-step solution

Identify the null and alternative hypothesis

The null hypothesis (\(H_0\)) is that there's no difference in mean satisfaction level among three interview techniques. The alternative hypothesis (\(H_1\)) is that there is a difference in mean satisfaction level among these techniques.

Conduct the ANOVA test

In this test, given that the \(F\) statistic is \(4.12\), the degrees of freedom for numerator (treatments) is \(k - 1 = 3 - 1 = 2\) and the degrees of freedom for denominator (error) is \(N - k = 33 - 3 = 30\). Here, \(k\) is the number of treatment groups and \(N\) is the total number of subjects. We conduct the level \(0.05\) test using these values.

Find the critical value

We look up the critical value for \(F\) in the \(F\)-distribution table for a \(.05\) significance level with \(2\) and \(30\) degrees of freedom. This value is approximately \(3.32\). If our computed \(F\) is greater than this, we reject the null hypothesis.

Compare the calculated F statistic with the critical value

As the given \(F\) statistic (\(4.12\)) is greater than the critical value (\(3.32\)), we reject the null hypothesis at level \(.05\), indicating that there is a difference in mean satisfaction level among these techniques.

Conclude for part (b)

In part (b), even though the sample size has increased to \(45\), the result of the test will remain the same since the \(F\) statistic (\(4.12\)) would still be greater than the critical value. Taking into account the number of tested subjects, we calculate the degrees of freedom as \(3 - 1 = 2\) for the treatments and \(45 - 3 = 42\) for the error. The F-distribution critical value for \(2\) and \(42\) degrees of freedom will be lower than the earlier critical value (\(3.32\)). Thus, the \(F\) statistic would remain significant, indicating that our conclusion from part (a) still holds.

Key concepts

Null Hypothesis

The null hypothesis, denoted as \(H_0\), is a fundamental principle in statistical testing, particularly in the ANOVA test. It posits that there's no effect or no difference and serves as a starting assumption for the test. In the context of our exercise, the null hypothesis states that there's no difference in mean satisfaction level across the three interviewing techniques.

Understanding the null hypothesis is crucial because it provides a baseline to which we compare the effects observed in our study. It's essentially the 'status quo' that we're challenging with our research. If statistical evidence is strong enough to reject the null hypothesis, then we can consider alternative possibilities, represented by the alternative hypothesis. Otherwise, we lack sufficient evidence to do so, and the null hypothesis stands.

Alternative Hypothesis

The alternative hypothesis, represented as \(H_1\) or \(H_a\), provides a contrast to the null hypothesis. In our ANOVA example, the alternative hypothesis suggests that there's at least one difference in mean satisfaction level between the interviewing techniques.

The alternative hypothesis is what researchers aim to support, based on their belief that a change, difference, or effect exists. In conducting an ANOVA test, the goal is to determine whether the observed differences in data are substantial enough to reject the null hypothesis in favor of the alternative hypothesis. It's important for students to grasp that the alternative hypothesis is not automatically accepted when the null is rejected, but rather it becomes more plausible.

F-distribution

The F-distribution is a particular statistical distribution critical for the ANOVA test. It's used to determine the critical value for comparing the calculated F statistic. The shape of the F-distribution is positively skewed and depends on two sets of degrees of freedom: one for the numerator (between-group variation) and another for the denominator (within-group variation).

Our exercise demonstrates how the F statistic, reported as \(4.12\), is compared to a critical value obtained from the F-distribution table. The significance of this comparison lies in decision-making: If the F statistic exceeds the critical value from the F-distribution table at a given level of significance (e.g., 0.05), we reject the null hypothesis. Understanding the concept of the F-distribution helps students appreciate why certain F values lead to rejecting the null hypothesis.

Degrees of Freedom

Degrees of freedom are a measure of the amount of independence in the data, used in various statistical tests, including the ANOVA. For ANOVA, degrees of freedom are split into two types: degrees of freedom for the numerator (associated with the variation among group means) and degrees of freedom for the denominator (associated with the variation within the groups).

In our exercise, the degrees of freedom for the numerator are calculated as the number of groups minus one \(k - 1\), while the degrees of freedom for the denominator are the total number of subjects minus the number of groups \(N - k\). Understanding how to properly calculate degrees of freedom is essential because they directly influence the critical F-value, affecting the outcome of the hypothesis test.