Question

Propagation of error. Suppose that you can measure independent variables \(x\) and \(y\) and that you have a dependent variable \(f(x, y)\). The average values are \(\bar{x}, \bar{y}\), and \(f\). We define the error in \(x\) as the deviations \(\varepsilon_{x}=x-\hat{x}\), in \(y\) as \(\varepsilon_{y}=y-\bar{y}\), and in \(f\) as \(\varepsilon_{f}=f-\bar{f}\). (a) Use a Taylor series expansion to express the error \(\varepsilon_{f}\) in \(f\), as a function of the errors \(\varepsilon_{x}\) and \(\varepsilon_{y}\) in \(x\) and \(y\). (b) Compute the mean-squared error \(\left\langle\varepsilon_{f}^{2}\right\rangle\) as a function of \(\left\langle e_{x}^{2}\right\rangle\) and \(\left(\varepsilon_{y}^{2}\right)\).

Short answer

The mean-squared error is \( \left\langle \varepsilon_{f}^{2} \right\rangle \approx \left( \frac{\partial f}{\partial x} \right)^{2} \left\langle \varepsilon_{x}^{2} \right\rangle + \left( \frac{\partial f}{\partial y} \right)^{2} \left\langle \varepsilon_{y}^{2} \right\rangle \)

Step-by-step solution

Understand the Problem

We are given independent variables \(x\) and \(y\) and a dependent variable \(f(x, y)\). The goal is to find the error \(\varepsilon_{f}\) in \(f\) and compute the mean-squared error \(\left\langle \varepsilon_{f}^{2} \right\rangle\). This will involve using a Taylor series expansion and propagation of errors.

Express \( \varepsilon_{f} \)

Using a Taylor series expansion near the average values \(\bar{x}\) and \(\bar{y}\), the error in \(f\) can be approximated as:\[\varepsilon_{f} \approx \frac{\partial f}{\partial x} \varepsilon_{x} + \frac{\partial f}{\partial y} \varepsilon_{y}\]

Mean-Squared Error Calculation

To find the mean-squared error \(\left\langle \varepsilon_{f}^{2} \right\rangle\), square the expression from Step 1 and take the expectation value:\[ \left\langle \varepsilon_{f}^{2} \right\rangle \approx \left\langle \left( \frac{\partial f}{\partial x} \varepsilon_{x} + \frac{\partial f}{\partial y} \varepsilon_{y}\right)^{2} \right\rangle \]

Expand the Squared Term

Expand the term inside the expectation:\[ \left\langle \varepsilon_{f}^{2} \right\rangle \approx \left\langle \left( \frac{\partial f}{\partial x} \varepsilon_{x} \right)^{2}\right\rangle + \left\langle\left( \frac{\partial f}{\partial y} \varepsilon_{y} \right)^{2} \right\rangle + 2 \left\langle \left( \frac{\partial f}{\partial x} \varepsilon_{x} \frac{\partial f}{\partial y} \varepsilon_{y}\right) \right\rangle \]

Simplify using Independence

Since \(\varepsilon_{x}\) and \(\varepsilon_{y}\) are independent, the cross term vanishes:\[ 2 \left\langle \frac{\partial f}{\partial x} \varepsilon_{x} \frac{\partial f}{\partial y} \varepsilon_{y} \right\rangle = 0 \]Thus,\[ \left\langle \varepsilon_{f}^{2} \right\rangle \approx \left( \frac{\partial f}{\partial x} \right)^{2} \left\langle \varepsilon_{x}^{2} \right\rangle + \left( \frac{\partial f}{\partial y} \right)^{2} \left\langle \varepsilon_{y}^{2} \right\rangle \]

Key concepts

Taylor Series Expansion

In order to approach the problem of error propagation in a system with independent variables, we employ the Taylor series expansion. The Taylor series allows us to approximate a function around a point by expressing it as a sum of its derivatives at that point. This expansion is particularly useful when we only need a linear approximation of our function.

For a function \(f(x, y)\) with mean values \(\bar{x}\) and \(\bar{y}\), we can express the small changes in \(f\) (denoted as \(\varepsilon_{f}\)) as a combination of small changes in \(x\) and \(y\) (denoted as \(\varepsilon_{x}\) and \(\varepsilon_{y}\) respectively). Using the first-order Taylor series expansion, we get:

\[\varepsilon_{f} \approx \frac{\partial f}{\partial x} \varepsilon_{x} + \frac{\partial f}{\partial y} \varepsilon_{y}\]
This equation tells us how the errors in the independent variables \(x\) and \(y\) contribute to the error in the dependent variable \(f\).

The partial derivatives \(\frac{\partial f}{\partial x}\) and \(\frac{\partial f}{\partial y}\) are evaluated at the average values \(\bar{x}\) and \(\bar{y}\). These derivatives provide the sensitivity of the function \(f\) to changes in \(x\) and \(y\).

Mean-Squared Error

The mean-squared error (MSE) quantifies the average of the squares of the errors. It is a measure of how much the estimated values differ from the actual mean values. To calculate the mean-squared error of \(f\) (\(\left\langle \varepsilon_{f}^{2} \right\rangle\)), we start by squaring the expression obtained from the Taylor series expansion and taking its expectation value:

\[ \left\langle \varepsilon_{f}^{2} \right\rangle \approx \left\langle \left( \frac{\partial f}{\partial x} \varepsilon_{x} + \frac{\partial f}{\partial y} \varepsilon_{y}\right)^{2} \right\rangle \]
Expanding the squared term gives us:

\[ \left\langle \varepsilon_{f}^{2} \right\rangle \approx \left\langle \left( \frac{\partial f}{\partial x} \varepsilon_{x} \right)^{2}\right\rangle + \left\langle\left( \frac{\partial f}{\partial y} \varepsilon_{y} \right)^{2} \right\rangle + 2 \left\langle \left( \frac{\partial f}{\partial x} \varepsilon_{x} \frac{\partial f}{\partial y} \varepsilon_{y}\right) \right\rangle \]
Since \(\varepsilon_{x}\) and \(\varepsilon_{y}\) are independent, their product's expectation value is zero:

\[ 2 \left\langle \frac{\partial f}{\partial x} \varepsilon_{x} \frac{\partial f}{\partial y} \varepsilon_{y} \right\rangle = 0 \]
Thus, we are left with:

\[ \left\langle \varepsilon_{f}^{2} \right\rangle \approx \left( \frac{\partial f}{\partial x} \right)^{2} \left\langle \varepsilon_{x}^{2} \right\rangle + \left( \frac{\partial f}{\partial y} \right)^{2} \left\langle \varepsilon_{y}^{2} \right\rangle \]
The equation shows that the mean-squared error in \(f\) is a sum of the contributions from the mean-squared errors in \(x\) and \(y\), weighted by the squares of the partial derivatives of \(f\). This insight is beneficial for understanding how the errors in independent variables propagate through an equation to affect the dependent variable.

Independent Variables

Independent variables play a crucial role in error propagation. They are the variables whose errors we are trying to understand and control.

Two variables, \(x\) and \(y\), are considered independent if the value of one does not affect the value of the other. In mathematical terms, if \(x\) and \(y\) are independent, the joint probability of \(x\) and \(y\) is the product of their individual probabilities. This property allows us to simplify the propagation of their errors through equations.

When computing the mean-squared error of the dependent variable \(f\), we used the fact that the cross-term involving \(\varepsilon_{x}\) and \(\varepsilon_{y}\) is zero. This zero cross-term results from the independence condition.

This simplification can be expressed as:
\[ 2 \left\langle \frac{\partial f}{\partial x} \varepsilon_{x} \frac{\partial f}{\partial y} \varepsilon_{y} \right\rangle = 0 \]
This shows how the errors from independent variables combine in a simple manner, without additional interaction terms, making it easier to understand the overall error behavior of the function \(f\).

Understanding the independence of variables is vital in fields like physics, engineering, and data science because it helps in designing experiments and models that minimize error and maximize accuracy.