Question

Let \(m_{1}, m_{2}, \cdots, m_{n}\) be a set of measurements, and define the values of \(x_{i}\) by \(x_{1}=\) \(m_{1}-a, x_{2}=m_{2}-a, \cdots, x_{n}=m_{n}-a,\) where \(a\) is some number (as yet unspecified, but the same for all \(x_{i}\) ). Show that in order to minimize \(\sum_{i=1}^{n} x_{i}^{2},\) we should choose \(a=(1 / n) \sum_{i=1}^{n} m_{i} .\) Hint: Differentiate \(\sum_{i=1}^{n} x_{i}^{2}\) with respect to \(a .\) You have shown that the arithmetic mean is the "best" average in the least squares sense, that is, that if the sum of the squares of the deviations of the measurements from their "average" is a minimum, the "average" is the arithmetic mean (rather than, say, the median or mode).

Short answer

To minimize \sum_{i=1}^{n} x_{i}^{2}, choose \(a = \frac{1}{n} \sum_{i=1}^{n} m_{i}\).

Step-by-step solution

Define the Problem

Given measurements are denoted as \(m_{1}, m_{2}, \cdots, m_{n}\). The new values \(x_{i}\) are defined by \(x_{1} = m_{1} - a, x_{2} = m_{2} - a, \cdots, x_{n} = m_{n} - a\). We need to show that to minimize \sum_{i=1}^n x_{i}^2, we should choose \(a = \frac{1}{n} \sum_{i=1}^n m_{i}\).\

Express the Sum of Squares

The sum of the squares of \(x_i\) is given by: \sum_{i=1}^{n} x_{i}^{2} = \sum_{i=1}^{n} (m_{i} - a)^2\.

Set Up the Function to Minimize

Define the function to minimize: \[f(a) = \sum_{i=1}^{n} (m_{i} - a)^2 .\]

Differentiate with Respect to a

Differentiate \(f(a)\) with respect to \(a\) to find the critical points: \frac{d}{da} f(a) = \frac{d}{da} \sum_{i=1}^{n} (m_{i} - a)^2 .\

Apply the Derivative

Using the chain rule, differentiate each term: \[ \frac{d}{da} (m_{i} - a)^2 = 2(m_{i} - a)(-1) = -2(m_{i} - a) \] Therefore, \[ \frac{d}{da} f(a) = \sum_{i=1}^{n} -2(m_{i} - a) = -2 \sum_{i=1}^{n} (m_{i} - a) .\]

Set the Derivative to Zero

To minimize \(f(a)\), set the derivative equal to zero: \[ -2 \sum_{i=1}^n (m_{i} - a) = 0 .\]

Solve for a

Solve the equation for \(a\): \[ \sum_{i=1}^n (m_{i} - a) = 0 .\] This simplifies to: \[ \sum_{i=1}^n m_{i} - na = 0 .\] Therefore: \[ a = \frac{1}{n} \sum_{i=1}^{n} m_{i} .\]

Conclusion

Thus, we have shown that to minimize \sum_{i=1}^{n} x_{i}^{2}, we should choose \(a = \frac{1}{n} \sum_{i=1}^{n} m_{i} .\) This choice of \(a\) is the arithmetic mean of the measurements.\

Key concepts

Arithmetic Mean

The arithmetic mean, often simply called the 'mean' or 'average', is a fundamental concept in statistics. It represents the central value of a set of numbers. To find the arithmetic mean of a set of measurements denoted as \(m_{1}, m_{2}, \cdots, m_{n}\), you sum all the measurements and divide by the number of measurements. Mathematically, it is expressed as \[a = \frac{1}{n} \sum_{i=1}^{n} m_{i}.\]

The importance of the arithmetic mean becomes evident when we try to minimize the sum of squared deviations from a chosen point. When we use the arithmetic mean, we minimize the sum of the squares of the deviations, explaining why it's often referred to as the 'best' average in this context. It's the average value that minimizes the variability of the data points from this central point.

In practical terms, the arithmetic mean provides a useful summary of the dataset, giving us a single value that represents the entire set of data points. This can be essential for comparing different datasets or understanding overall trends within the data.

Differentiation

Differentiation is a vital tool in calculus used to determine the rate at which a function is changing at any given point. It's symbolized by the derivative, \frac{d}{dx}\, which provides the slope of the function's graph at a specific point.

In the context of minimizing the sum of the squared differences, differentiation is used to find the critical points of the function we want to minimize. The function we consider is \[f(a) = \sum_{i=1}^{n} (m_{i} - a)^2.\]

We apply differentiation to this function with respect to \a\, which involves using the chain rule. Simply, the chain rule here helps us manage the composite nature of the function. Differentiating \f(a)\ with respect to \a\ gives us \-2 \sum_{i=1}^{n} (m_{i} - a)\. Setting this result to zero allows us to solve for \a\, identifying the point where the sum of the squared deviations is minimized.

This process is crucial because it systematically leads to finding the optimal value of \a\. It reveals that the arithmetic mean minimizes the sum of squared deviations, thus differentiating ensures we accurately find the point of minimization.

Minimization

Minimization, in the scope of optimization, refers to the process of finding the point or value where a given function reaches its smallest value. When we talk about minimization in least squares, we aim to find the value that minimizes the sum of squared differences between observed values and a chosen point.

In our exercise, we start by defining a function representing the sum of squared deviations: \[f(a) = \sum_{i=1}^{n} (m_{i} - a)^2.\]

We then use differentiation to find where the derivative of this function is zero, as this will typically indicate a minimum. Setting the derivative equal to zero gives us \-2 \sum_{i=1}^{n} (m_{i} - a) = 0.\ From here, we solve for \a\, leading us to \a = \frac{1}{n} \sum_{i=1}^{n} m_{i}.\

Thus, the process of minimization confirmed that the arithmetic mean is the value that reduces the sum of the squared deviations to its least possible value. This methodological approach is not only applicable to statistical data but is a fundamental aspect of numerous optimization problems in different fields like economics, engineering, and machine learning.