Question

Define the sample variance by \(s^{2}=(1 / n) \sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2} .\) Show that the expected value of \(s^{2}\) is \([(n-1) / n] \sigma^{2} .\) Hints: Write $$ \begin{aligned} \left(x_{i}-\bar{x}\right)^{2} &=\left[\left(x_{i}-\mu\right)-(\bar{x}-\mu)\right]^{2} \\ &=\left(x_{i}-\mu\right)^{2}-2\left(x_{i}-\mu\right)(\bar{x}-\mu)+(\bar{x}-\mu)^{2} \end{aligned} $$ Find the average value of the first term from the definition of \(\sigma^{2}\) and the average value of the third term from Problem 2, To find the average value of the middle term write $$ (\bar{x}-\mu)=\left(\frac{x_{1}+x_{2}+\cdots+x_{n}}{n}-\mu\right)=\frac{1}{n}\left[\left(x_{1}-\mu\right)+\left(x_{2}-\mu\right)+\cdots+\left(x_{n}-\mu\right)\right] $$ show by Problem \(7.12\) that $$ E\left[\left(x_{i}-\mu\right)\left(x_{j}-\mu\right)\right]=E\left(x_{i}-\mu\right) E\left(x_{j}-\mu\right)=0 \quad \text { for } \quad t \neq j $$ and evaluate \(E\left[\left(x_{i}-\mu\right)^{2}\right]\) (same as the first term). Collect terms to find $$ E\left(s^{2}\right)=\frac{n-1}{n} \sigma^{2} $$

Short answer

The expected value of the sample variance is \( \frac{n-1}{n} \sigma^{2} \).

Step-by-step solution

Understanding the Sample Variance

The sample variance formula is given by \[ s^{2} = \frac{1}{n} \, \sum_{i=1}^{n} \left( x_{i} - \bar{x} \right)^{2}. \] Let's show that the expected value of \( s^{2} \) is \( \frac{(n-1)}{n} \sigma^{2} \).

Expressing \( (x_{i} - \bar{x})^{2} \)

We have: \[ (x_{i} - \bar{x})^{2} = \left[ (x_{i} - \mu) - (\bar{x} - \mu) \right]^{2}. \] Expanding this, we obtain: \[ (x_{i} - \bar{x})^{2} = (x_{i} - \mu)^{2} - 2(x_{i} - \mu)(\bar{x} - \mu) + (\bar{x} - \mu)^{2}. \]

Expected Value of Each Term

We need the average value of these terms: 1. The first term's expected value is \( E[(x_{i} - \mu)^{2}] \) which is equal to \( \sigma^{2} \). 2. The second term's expected value is 0 because \[ E[(x_{i} - \mu)(\bar{x} - \mu)] = 0 \text{ for } i eq j \text{ (using Problem 7.12)}. \] 3. The third term's expected value is \[ E[(\bar{x} - \mu)^{2}] = \frac{\sigma^{2}}{n}. \]

Summing the Expected Values

Sum the expected values of the expanded form: \[ E[(x_{i} - \bar{x})^{2}] = \sigma^{2} - 0 + \frac{\sigma^{2}}{n} = \sigma^{2} \left(1 - \frac{1}{n}\right) + \frac{\sigma^{2}}{n} = \left(1 - \frac{1}{n}\right)\sigma^{2}. \]

Finding the Expected Value of \( s^{2} \)

Now, the expected value of sample variance is: \[ E[s^{2}] = \frac{1}{n} E\left[ \sum_{i=1}^{n} (x_{i} - \bar{x})^{2} \right] = \frac{1}{n}\sum_{i=1}^{n} E[(x_{i} - \bar{x})^{2}] \] Given the expectation for each term: \[ E[s^{2}] = \frac{1}{n} \cdot n \cdot \left(1 - \frac{1}{n}\right) \sigma^{2} = \left(1 - \frac{1}{n}\right) \sigma^{2} = \frac{n-1}{n} \sigma^{2}. \]

Key concepts

Sample Variance

In statistics, the sample variance is a measure of how spread out the values in a sample are around the mean. It's calculated using the formula: \[ s^{2} = \frac{1}{n} \sum_{i=1}^{n} \left( x_{i} - \bar{x} \right)^{2} \] where:

• \(s^{2}\) is the sample variance.
• \(n\) is the number of samples.
• \(x_{i}\) represents each individual value in the sample.
• \(\bar{x}\) is the sample mean.

The sample variance helps us understand how much the values in our sample differ from the average value. It's an important concept because it provides insight into the variability of the data.

Expected Value

The expected value, often denoted by \(E[X]\), is a fundamental concept in probability and statistics. It's the long-run average value of repetitions of the experiment it represents. For a sample variance, the expected value is a measure of the average of all possible sample variances from the population.

Mathematically, for the sample variance \(s^{2}\), the expected value is shown as:
\[ E[s^{2}] = \frac{n-1}{n} \sigma^{2} \]
In this exercise, we demonstrated that calculation step-by-step. By rewriting the equation and measuring every term’s contribution, we found that the expected value of the sample variance is slightly less than the actual population variance \(\sigma^{2}\). This bias adjustment is crucial for making accurate predictions when using sample data.

Standard Deviation

The standard deviation is another key concept in statistics, closely related to variance. It provides a measure of the dispersion or variability in a data set and is the square root of the variance.

The formula for standard deviation is:
\[ \sigma = \sqrt{s^{2}} \]
When calculating standard deviation, understanding variance is essential because the standard deviation is derived directly from it. While variance is useful for mathematical calculations due to its aggregation properties, standard deviation is more interpretable as it shares the same unit as the data, giving a more intuitive sense of spread.

Probability Theory

Probability theory is the branch of mathematics concerned with analyzing random phenomena. It provides the tools necessary to model and predict outcomes for random processes. Key concepts within probability theory include random variables, expected values, variance, and distribution functions.

Understanding these concepts is essential for interpreting results from statistical analyses, such as the sample variance and its expected value. For instance, in this exercise, probability theory explains why the expected value of the sample variance formula compensates for the sample size by the factor \(\frac{n-1}{n}\).

By grasping probability theory, one can better understand and predict the behavior of data under uncertainty, which is the foundation of making statistically sound decisions.