Question

Prove the least squares approximation property of Legendre polynomials [see (9.5) and (9.6)] as follows. Let \(f(x)\) be the given function to be approximated. Let the functions \(p_{l}(x)\) be the normalized Legendre polynomials, that is, $$p_{l}(x)=\sqrt{\frac{2 l+1}{2}} P_{l}(x) \quad \text { so that } \quad \int_{-1}^{1}\left[p_{l}(x)\right]^{2} d x=1$$ Show that the Legendre series for \(f(x)\) as far as the \(p_{2}(x)\) term is $$f(x)=c_{0} p_{0}(x)+c_{1} p_{1}(x)+c_{2} p_{2}(x) \quad \text { with } \quad c_{l}=\int_{-1}^{1} f(x) p_{l}(x) d x$$ Write the quadratic polynomial satisfying the least squares condition as \(b_{0} p_{0}(x)+\) \(b_{1} p_{1}(x)+b_{2} p_{2}(x)\) (by Problem 5.14 any quadratic polynomial can be written in this form). The problem is to find \(b_{0}, b_{1}, b_{2}\) so that $$I=\int_{-1}^{1}\left[f(x)-\left(b_{0} p_{0}(x)+b_{1} p_{1}(x)+b_{2} p_{2}(x)\right)\right]^{2} d x$$ is a minimum. Square the bracket and write \(I\) as a sum of integrals of the individual terms. Show that some of the integrals are zero by orthogonality, some are 1 because the \(p_{t}\) 's are normalized, and others are equal to the coefficients \(c_{l}\). Add and subtract \(c_{0}^{2}+c_{1}^{2}+c_{2}^{2}\) and show that $$I=\int_{-1}^{1}\left[f^{2}(x)+\left(b_{0}-c_{0}\right)^{2}+\left(b_{1}-c_{1}\right)^{2}+\left(b_{2}-c_{2}\right)^{2}-c_{0}^{2}-c_{1}^{2}-c_{2}^{2}\right] d x$$ Now determine the values of the \(b\) 's to make \(I\) as small as possible. (Hint: The smallest value the square of a real number can have is zero.) Generalize the proof to polynomials of degree \(n\).

Short answer

To minimize the integral and get the least squares approximation, set \( b_{l} = c_{l} \) for \( l = 0, 1, 2 \). Generalize for degree \( n \).

Step-by-step solution

- Understand the Legendre Series Representation

The Legendre series for a function \[f(x) = c_{0} p_{0}(x) + c_{1} p_{1}(x) + c_{2} p_{2}(x)\] provides an approximation using the Legendre polynomials. Where each coefficient is given by: \[c_{l} = \int_{-1}^{1} f(x) p_{l}(x) \, dx\].

- Write the Quadratic Polynomial

Write the quadratic polynomial satisfying the least squares condition as:\[b_{0} p_{0}(x) + b_{1} p_{1}(x) + b_{2} p_{2}(x)\].

- Formulate the Integral to Minimize

To minimize the error, define the integral \[I = \int_{-1}^{1} \left[ f(x) - \left( b_{0} p_{0}(x) + b_{1} p_{1}(x) + b_{2} p_{2}(x) \right) \right]^2 dx\].

- Expand the Squared Term Inside the Integral

Expand the integrand:\[I = \int_{-1}^{1} \left[ f(x)^2 - 2f(x)(b_{0} p_{0}(x) + b_{1} p_{1}(x) + b_{2} p_{2}(x)) + (b_{0} p_{0}(x) + b_{1} p_{1}(x) + b_{2} p_{2}(x))^2 \right] dx\].

- Simplify Using Orthogonality

Apply orthogonality of Legendre polynomials. Non-diagonal terms vanish:\[I = \int_{-1}^{1} f(x)^2 dx - 2\sum_{l=0}^{2} b_{l}\int_{-1}^{1} f(x)p_{l}(x) dx + \sum_{l=0}^{2} b_{l}^2 \int_{-1}^{1} [p_{l}(x)]^2 dx\].Since \( \int_{-1}^{1} [p_{l}(x)]^2 dx = 1\), we get: \[I = \int_{-1}^{1} f(x)^2 dx - 2\sum_{l=0}^{2} b_{l} c_{l} + \sum_{l=0}^{2} b_{l}^2\].

- Add and Subtract Terms

Rearrange and complete the square:\[I = \int_{-1}^{1} \left[f(x)^2 + \sum_{l=0}^{2} \left(b_{l} - c_{l}\right)^2 - \sum_{l=0}^{2} c_{l}^2\right] dx\].

- Minimize the Integral

To minimize \(I\), the expression \(\left(b_{0}-c_{0}\right)^2 + \left(b_{1}-c_{1}\right)^2 + \left(b_{2}-c_{2}\right)^2\)must be zero. Hence, \(b_{l} = c_{l}\) for all \(l = 0, 1, 2\).

- Generalize to Polynomials of Degree n

The same approach applies to polynomials of degree \(n\). The least squares approximation polynomial of degree \(n\) will be \[f(x) \approx \sum_{l=0}^{n} c_{l} p_{l}(x)\] with coefficients \(c_{l} = \int_{-1}^{1} f(x) p_{l}(x) dx\).

Key concepts

Legendre Polynomials

Legendre polynomials are a series of orthogonal polynomials that are solutions to Legendre's differential equation. They are widely used in approximation theory, physics, and engineering because of their useful properties. In this topic, we deal with normalized versions of these polynomials, represented as:

• p_0(x) = \text{normalized } P_0(x)
• p_1(x) = \text{normalized } P_1(x)
• p_2(x) = \text{normalized } P_2(x)

Normalized Legendre polynomials are adjusted so that their norm is unity over the interval \texttt{[-1, 1]}, i.e., \[\texttt{\textbackslash int\textbackslash_-1\textbackslash\textcaret1\textbackslash left[p\textbackslash_l(x)\textbackslash right]\textbackslash caret2\textbackslash dx = 1.\]\When approximating a function, these polynomials can make the process of finding coefficients simpler and more efficient due to their orthagonality.

Orthogonality

Orthogonality is a key concept in many areas of mathematics, especially in functional analysis and approximation theory. Legendre polynomials possess this property, which simplifies the computation of coefficients in polynomial approximations. Specifically, two functions \( p_i(x) \) and \( p_j(x) \) are orthogonal over an interval \texttt{[-1, 1]} if:\[\texttt{\textbackslash int\textbackslash_-1\textbackslash\textcaret1 p\textbackslash_i(x) p\textbackslash_j(x)\textbackslash dx = 0 \text{ for } i eq j.\]\This property helps simplify the integral when finding the coefficients for polynomial approximation. Since the orthogonal terms vanish and the integrals of the squared terms are one, it becomes straightforward to calculate each coefficient by integrating against the function \( f(x) \) being approximated.

In our exercise, this simplification allows us to rewrite the integral I and cancel out many terms, leaving us with a summed squared difference for minimization.

Polynomial Approximation

Polynomial approximation involves approximating a complex or unknown function with a polynomial. The goal is to find a polynomial that minimizes the difference between it and the function over a specific interval. Legendre polynomials are particularly useful for this because of their orthogonality and normalization. The polynomial approximation of a function \(f(x)\) using Legendre polynomials up to degree 2 is given by:

• \[f(x) \text{ \textapprox } c\textbackslash_0 p\textbackslash_0(x) + c\textbackslash_1 p\textbackslash_1(x) + c\textbackslash_2 p\textbackslash_2(x).\]\

Here, \(c_l\) are the coefficients calculated through integration. By using the least squares method, we aim to find coefficients such that the integral of the squared difference between the function and its approximation is minimized over \texttt{[-1, 1]}. This results in a polynomial that is a best fit in the least-squares sense.

In the exercise, we show that the best-fitting quadratic polynomial can be written in terms of the first three normalized Legendre polynomials, and we formulate and minimize a specific integral to find the coefficients that satisfy this condition.

Coefficient Calculation

The coefficients \( c_l \) for the polynomial approximation are found by projecting the function \( f(x) \) onto each of the normalized Legendre polynomials \( p_l(x) \). This projection is achieved via the integral:

• \[c_l = \texttt{\textbackslash int\textbackslash_-1\textbackslash\textcaret1 f(x) p\textbackslash_l(x) \textbackslash dx}.\]\

To find these coefficients, we need to:

• Multiply \( f(x) \) by the given Legendre polynomial \( p_l(x) \).
• Integrate the product over the interval \texttt{[-1, 1]}.
• Repeat this process for each desired degree of polynomial.
  

In the least squares sense, these coefficients ensure that the polynomial approximation is the closest possible match to \( f(x) \) by minimizing the error measure defined earlier.

In our exercise, we do precisely this by deriving expressions for \( b_l \), rearranging terms, and showing they minimize the error integral \( I \). We then generalize this approach to polynomials of any degree \( n \), using the same fundamental process.