Question

It is apparent that the error \(e_{s}\) in Table \(14.2\) is only first order. But why is this necessarily so? More generally, let \(f(x)\) be smooth with \(f^{\prime \prime}\left(x_{0}\right) \neq 0\). Show that the truncation error in the formula $$ f^{\prime}\left(x_{0}\right) \approx \frac{f\left(x_{1}\right)-f\left(x_{-1}\right)}{h_{0}+h_{1}} $$ with \(h_{1}=h\) and \(h_{0}=h / 2\) must decrease linearly, and not faster, as \(h \rightarrow 0\).

Short answer

Answer: The truncation error is first-order for the given formula because, as we analyzed using Taylor series expansion and calculated the truncation error, it was found to be proportional to the square of the step size h. When h approaches 0, the truncation error decreases linearly and not faster, indicating the first-order nature of the error.

Step-by-step solution

Taylor series expansion

Let's expand the function \(f(x)\) using the Taylor series at the point \(x_0\), keeping up to the second derivative term. We expand the function at both points \(x_1\) and \(x_{-1}\): $$ f(x_1) = f(x_0) + h_1f'(x_0) + \frac{1}{2}h_1^2f''(x_0) + R_1(x_1) $$ and $$ f(x_{-1}) = f(x_0) - h_0f'(x_0) + \frac{1}{2}h_0^2f''(x_0) + R_{-1}(x_{-1}) $$ where \(R_1(x_1)\) and \(R_{-1}(x_{-1})\) are the remainder terms containing higher-order derivatives.

Solve for \(f'(x_0)\)

Now we will solve for \(f'\) at \(x_0\) from the Taylor series expansions of \(f(x_1)\) and \(f(x_{-1})\): Subtract the second equation from the first equation: $$ f(x_1) - f(x_{-1}) = (h_0 + h_1)f'(x_0) + \frac{1}{2}(h_1^2 - h_0^2)f''(x_0) + R_1(x_1) - R_{-1}(x_{-1}) $$ Now let's solve for \(f'(x_0)\): $$ f'(x_0) \approx \frac{f(x_1) - f(x_{-1}) - \frac{1}{2}(h_1^2 - h_0^2)f''(x_0) - (R_1(x_1) - R_{-1}(x_{-1}))}{h_0 + h_1} $$

Calculate the truncation error

To calculate the truncation error, we need to find the difference between the true value of the derivative \(f'(x_0)\) and the approximation given by the formula: $$ e_s = \frac{1}{2}(h_1^2 - h_0^2)f''(x_0) + R_1(x_1) - R_{-1}(x_{-1}) $$ Notice that as \(h \rightarrow 0\), \(h_0\) and \(h_1\) will also approach 0. Therefore, the remainder terms \(R_1(x_1)\) and \(R_{-1}(x_{-1})\) containing higher-order derivatives will go to 0 faster than the term \((\frac{1}{2}(h_1^2 - h_0^2)f''(x_0))\) since both terms will have at least a cubic power of \(h\).

Analyze the convergence rate

As \(h \rightarrow 0\), the truncation error becomes: $$ e_s \approx \frac{1}{2}(h_1^2 - h_0^2)f''(x_0) $$ Since \(h_1 = h\) and \(h_0 = \frac{h}{2}\), we get: $$ e_s \approx \frac{1}{2}(h^2 - \frac{h^2}{4})f''(x_0) = \frac{1}{4}h^2f''(x_0) $$ As we can see, the truncation error is proportional to the square of the step size \(h\). Therefore, as \(h \rightarrow 0\), the truncation error decreases linearly, and not faster. This shows that the error is first-order.

Key concepts

Truncation Error

In numerical differentiation, truncation error is crucial to consider as it reflects the difference between the actual mathematical function and its numerical approximation. When we use numerical methods, like finite differences, to estimate derivatives, we approximate continuous mathematical problems with discrete data points. This approximation leads to truncation error.

In the given exercise, the formula for the numerical differentiation approximation is:

• \[f'(x_0) \approx \frac{f(x_1) - f(x_{-1})}{h_0 + h_1}\]

The truncation error depends mostly on the higher-order terms that are neglected from the Taylor Series expansion of a function. In our specific example, when we expanded the function, leftover terms beyond the second derivative create the truncation error.

For small step sizes \(h\), the truncation error of the approximation decreases, but not at a rate faster than \(h\), making it a first-order method. This means the discrepancy diminishes linearly as \(h\) approaches zero.

Taylor Series Expansion

The Taylor Series expansion is a powerful tool in numerical analysis that approximates functions through an infinite sum of terms calculated from the values of its derivatives at a single point. This method is essential for deriving numerical differentiation formulas.

When approximating the derivative \(f'(x_0)\), we expand the function using Taylor Series at the points \(x_1\) and \(x_{-1}\), yielding:

• \[f(x_1) = f(x_0) + h_1f'(x_0) + \frac{1}{2}h_1^2f''(x_0) + R_1(x_1)\]
• \[f(x_{-1}) = f(x_0) - h_0f'(x_0) + \frac{1}{2}h_0^2f''(x_0) + R_{-1}(x_{-1})\]

Where \(R_1(x_1)\) and \(R_{-1}(x_{-1})\) are the remainder terms.

The remainder terms contain higher derivative terms of the function and contribute to the error of our approximation if not considered. By manipulating these expansions, we can solve for \(f'(x_0)\) and thus derive an expression for both the approximation and its error. This methodology is fundamental to understanding the limitations and range of accuracy of numerical methods.

Convergence Rate

Convergence rate in numerical analysis indicates how quickly a sequence approaches its limit, especially as the step size \(h\) reduces in our context. For the numerical differentiation formula in question, we analyze the convergence rate by examining how the truncation error decreases as \(h\) becomes smaller.

From our derivation, we found:

• As \(h \rightarrow 0\), the truncation error \(e_s \approx \frac{1}{4}h^2f''(x_0)\)

This result indicates that the error is a quadratic term, \(h^2\), suggesting a second-order error behavior with respect to \(h\). However, since this exercise is framed within a first-order error scenario, what is essential to observe is how linear approximation, \(\propto h\), shows the convergence rate effectively being controlled by the leading error term.

As \(h\) decreases towards zero, we observe a convergence behavior where the error diminishes linearly to zero. Thus, the method is said to have a linear convergence rate, consistent with the nomenclature of a first-order method.