Question

If you are a fan of complex arithmetic, then you will like this exercise. Suppose that \(f(z)\) is infinitely smooth on the complex plane and \(f(z)\) is real when \(z\) is real. We wish to approximate \(f^{\prime}\left(x_{0}\right)\) for a given real argument \(x_{0}\) as usual. (a) Let \(h>0\), assumed small. Show by a Taylor expansion of \(f\left(x_{0}+i h\right)\) about \(x_{0}\) that $$ f^{\prime}\left(x_{0}\right)=\Im\left[f\left(x_{0}+t h\right)\right] / h+\mathcal{O}\left(h^{2}\right) $$ Thus, a second order difference formula is obtained that does not suffer the cancellation error that plagues all the methods in Sections \(14.1-14.3\). (b) Show that, furthermore, the leading term of the truncation error is the same as that of the centered formula that stars in Examples \(14.1\) and \(14.7\).

Short answer

Question: Provide the second-order difference formula obtained in part (a). Answer: The second-order difference formula obtained in part (a) is given by: $$ f'(x_0) = \frac{\Im[f(x_0 + ih)]}{h} + \mathcal{O}(h^2) $$

Step-by-step solution

Part (a): Finding the second order difference formula

Let's start by expanding \(f(x_0 + ih)\) as a Taylor series around \(x_0\): $$ f(x_0 + ih) = f(x_0) + ihf'(x_0) - h^2\frac{f''(x_0)}{2!} - ih^3\frac{f'''(x_0)}{3!} + \cdots $$ Now we know that \(f(z)\) is real when \(z\) is real, so \(f(x_0)\) and \(f''(x_0)\) are real. We also have \(f(x_0+ih)\) as a complex number and want to find \(Im[f(x_0+ih)]\) which is given by: $$ \Im[f(x_0 + ih)] = h \cdot Im[f'(x_0)] - h^3\frac{1}{3!}\cdot Im[f'''(x_0)] + \cdots $$ We want to express this in terms of the derivative \(f'(x_0)\). To do that, note that \(f'(x_0)= \Re[f'(x_0)] + i\Im[f'(x_0)]\), so \(\Im[f'(x_0)]=\dfrac{\Im[f(x_0+ih)]}{h} + \mathcal{O}(h^2)\): $$ f'(x_0) = \frac{\Im[f(x_0 + ih)]}{h} + \mathcal{O}(h^2) $$

Part (b): Compare the truncation error with the centered formula

Now we will compare the truncation error of our obtained formula with the centered formula given as: $$ f'(x_0) = \frac{f(x_0 + h) - f(x_0 - h)}{2h} + \mathcal{O}(h^2) $$ For the centered formula, we expand in Taylor series \(f(x_0 + h)\) and \(f(x_0 - h)\) and subtract them: $$ f'(x_0) = \frac{2hf'(x_0) + h^3\frac{f'''(x_0)}{3!} + \cdots - (h^3\frac{f'''(x_0)}{3!} + \cdots)}{2h} + \mathcal{O}(h^2) $$ This simplifies to give the truncation error term of centered formula as: $$ f'(x_0) = f'(x_0) + h^2\frac{f'''(x_0)}{6} + \mathcal{O}(h^3) $$ Comparing the leading truncation error term of our formula and the centered formula: $$ \frac{\Im[f(x_0 + ih)]}{h} + \mathcal{O}(h^2) = f'(x_0) + h^2\frac{f'''(x_0)}{6} + \mathcal{O}(h^3) $$ thus, we see that the leading term of the truncation error in our formula is the same as that of the centered formula.

Key concepts

Taylor Expansion

Taylor expansion is a powerful tool in complex analysis and calculus, allowing us to estimate the values of complex functions around a specific point. In the context of our exercise, we used Taylor expansion to approximate the complex function \( f(x_0 + ih) \) near the point \( x_0 \). By doing so, we expressed the function as an infinite sum of terms derived from the function's derivatives at \( x_0 \).

• The general form of a Taylor series for a function \( f \) around the point \( x_0 \) can be written as:
  \[ f(x_0 + ih) = f(x_0) + ihf'(x_0) + \frac{(ih)^2}{2!}f''(x_0) + \cdots \]
• Each term in this expansion involves a higher-order derivative of the function, multiplied by powers of \( ih \), where \( h \) is a small difference intended to be very small so higher-order terms can be neglected.
  
• In our specific case, since the function \( f \) is real-valued at real points, some derivatives, particularly those of even order like \( f''(x_0) \), remain real.

By isolating the imaginary part of the Taylor expansion, we can derive useful approximations of derivatives, leading to robust difference formulas that are less prone to numerical cancellation errors.

Truncation Error

In numerical analysis, truncation error arises when an infinite series or integral is approximated by truncating it and using only a finite number of terms. This error is crucial to understand since it affects the accuracy of any numerical computation.

• In our exercise, truncation error plays a key role when approximating \( f'(x_0) \) using the expression derived through Taylor expansion.
  
• The formula was shown as \( f'(x_0) = \frac{\Im[f(x_0 + ih)]}{h} + \mathcal{O}(h^2) \). The Term \( \mathcal{O}(h^2) \) represents the order of the truncation error, indicating that the error goes to zero faster than \( h \) but slower than \( h^2 \).


• This truncation error is similar to the error seen in the centered difference formula, which also achieves an \( \mathcal{O}(h^2) \) order of accuracy. The key takeaway is that despite using only one term, we maintain a second-order accuracy.

Accurately accounting for and minimizing truncation error is essential in designing effective numerical methods. Understanding the nature of the truncation error helps in predicting the accuracy and behavior of differential approximations.

Difference Formulas

Difference formulas are essential in numerical differentiation, providing ways to estimate derivatives using function values. Our exercise illustrates a novel second-order difference formula tailored for complex functions, related closely to the concept of finite differences.

• Difference formulas, such as the forward, backward, and centered differences, are used to approximate derivatives for real functions. They utilize discrete function values at specific points to make these approximations.
  
• The second order difference formula derived here, \( f'(x_0) = \frac{\Im[f(x_0 + ih)]}{h} + \mathcal{O}(h^2) \), employs the imaginary part of the function to achieve a precise estimate of derivatives.
  
• This approach avoids certain complications like cancellation errors associated with other difference formulas, ensuring more stability and reliability in computational practices.

By harnessing Taylor expansions and tailoring our approach to complex functions, we developed a powerful technique for derivative approximation, overcoming typical challenges seen in traditional methods.