Question

Stirling approximation revisited The Stirling approximation is useful in a variety of differ. ent settings. The goal of the present problem is to work through a more sophisticated treatment of this approximation than the simple heuristic argument given in the chapter. Our task is to find useful representations of \(n\) then since terms of the form in \(n\) arise often in reasoning about entropy. (a) Begin by showing that $$n !=\int_{0}^{\infty} x^{n} \mathrm{e}^{-x} \mathrm{d} x$$ To demonstrate this, use repeated integration by parts. In particular, demonstrate the recurrence relation $$\int_{0}^{\infty} x^{n} e^{-x} d x=n \int_{0}^{\infty} x^{n-1} e^{-x} d x$$ and then argue that repeated application of this relation Ieads to the desired result. (b) Make plots of the integrand \(x^{n} e^{-x}\) for various values of \(n\) and observe the peak width and height of this inter. grand. We are interested now in finding the value of \(x\) for which this function is a maximum. The idea is that we will then expand about that maximum. To carry out this step, consider \(\ln \left(x^{\pi} e^{-x}\right)\) and find its maximum - argue why it is acceptable to use the logarithm of the original function as a surrogate for the function itself-that is, show that the maxima of both the function and its logarithm are at the same \(x\). Also, argue why it might be a good idea to use the logarithm of the integrand rather than the integrand itself as the basis of our analysis. Call the value of \(x\) for which this function is maximized \(x_{0}\). Now expand the logarithm about \(x_{0},\) In particular, examine $$\ln \left|\left(x_{0}+b\right)^{n} e^{-\left(x_{0}+b\right)}\right|=n \ln \left(x_{0}+d\right)-\left(x_{0}+8\right)$$ and expand to second order in \(\delta\), Exponentiate your result and you should now have an approximation to the orig. inal integrand which ts good in the neighborhood of \(x_{0}\) Plug this back into the integral (be careful with limits of Integration) and by showing that it is acceptable to send the lower limit of integration to \(-\infty,\) show that $$n ! \Leftrightarrow n^{\pi} \mathrm{e}^{-n} \int_{-\infty}^{\infty} \mathrm{e}^{-g^{2} / 2 n} \mathrm{d} \delta$$ Evaluate the integral and show that in this approximation $$n !=n^{n} e^{-n}(2 \pi n)^{1 / 2}$$ Also, take the logarithm of this result and make an argument as to why most of the time we can get away with dropping the \((2 \pi n)^{1 / 2}\) term.

Short answer

Finally after performing all the steps, it is found that the Stirling’s approximation for \(n!\) is \(n!=n^n e^{-n}(2 \pi n)^{1 / 2}\). Furthermore, by taking logarithm to this result, it is seen that the \((2\pi n)^{1 / 2}\) term often can be neglected due to its slower growth rate compared to other terms.

Step-by-step solution

Recursive integral representation

We are asked to demonstrate that \(n! = \int_{0}^{\infty} x^n e^{-x} dx\). We start off using integration by parts with \(u=x^n\), \(v=e^{-x}\), \(dv=-e^{-x} dx\), and \(du=n x^{n-1} dx\). After performing the integration by parts, we find a relation that can be recursively applied since after each integration by parts operation, the power of \(x\) in the integrand decreases by one. This provides an equivalent representation of \(n!\), and the base case corresponds to \(0!=\int_{0}^{\infty} x^0 e^{-x} dx\) which evaluates to 1, verifying our start point.

Examine properties of integrand

The next objective is to examine \(x^n e^{-x}\) for various values of \(n\), focusing on its peak width and height. Plot this function for different \(n\)'s to visually understand its behaviour. Find the maximum of this function or equivalently the maximum of \(\ln \left(x^n e^{-x}\right)\) which would be at the same \(x\) value. It is easier to work with logarithm of the function, because logarithm converts the product of functions to their sum, simplifying the derivative computation for finding maximum.

Expansion around the maximum

Call the value of \(x\) for which the logarithm function is maximized \(x_0\). This function is now expanded about \(x_0\). Consider the function \(\ln |(x_0+\delta)^n e^{-(x_0+\delta)}|=n \ln (x_0+\delta)-(x_0+\delta)\). This function should be expanded to the second order in \(\delta\). Hence, the approximation to the original integrand would be valid around \(x_0\).

Integral Evaluation

Substituting this approximation back into integral would be the next step. One should check the conditions under which the lower limit of integration can be extended to \(-\infty\). The eventual goal is to show that \(n ! \Leftrightarrow n^n e^{-n} \int_{-\infty}^{\infty} e^{-\delta^{2} / 2n} d\delta\). Evaluate this integral and achieve the Stirling’s approximation for \(n!\).

Simplify Result

The final approximated result would be \(n!=n^n e^{-n}(2 \pi n)^{1 / 2}\). Take the logarithm of this result and analyse why most of the times we can ignore the \((2\pi n)^{1 / 2}\) term.

Key concepts

Integration by Parts

Integration by parts is a powerful technique used in calculus to evaluate more complex integrals by decreasing the degree of the function inside the integral. This technique is based on the product rule of differentiation and can be summarized with the formula:
\[ \int u \ dv = uv - \int v \ du \]
This means, if you have a product of two functions, you can differentiate one part (the function designated as \(u\)) and integrate the other part (the function designated as \(dv\)).
In the exercise for demonstrating that the factorial \(n! = \int_{0}^{\infty} x^n e^{-x} dx\), integration by parts is used repeatedly. The choices made are \(u = x^n\) and \(dv = e^{-x} dx\), which transforms the integral and reduces the power of \(x\). Each repetition of these steps transforms the integral into a simpler form until it equals one at the base case (for \(0!\)).
Understanding the mechanism of integration by parts helps to simplify complex expressions, particularly when a direct approach to integration is not apparent.

Maximum of a Function

Finding the maximum of a function is a fundamental calculus problem often solved by taking the derivative of the function and setting it to zero. This identifies the critical points, which can then be analyzed to determine if they are maxima, minima, or saddle points.
In the context of the exercise, the goal is to find the maximum value of the function \(x^n e^{-x}\). By taking the derivative with respect to \(x\) and setting it equal to zero, the critical points are identified. However, instead of working directly with the function, its logarithm is used, thanks to the property of logarithms that makes derivatives easier to handle.
This substitution is valid because the logarithm function has the same critical points as the original function when dealing with positive values of \(x\). Calculating the maximum using \(\ln(x^n e^{-x})\) simplifies the problem, turning the multiplication into addition, which substantially eases finding the derivative and thus the critical points.

Logarithmic Expansion

Logarithmic expansion is a technique that simplifies the handling of functions by expanding around a point, usually to make the solution of a problem more manageable.
In this exercise, after finding the point \(x_0\) where \(\ln(x^n e^{-x})\) reaches its maximum, an expansion around this point simplifies the function for further analysis.
This process typically involves finding the first few terms in the Taylor series expansion, capitalizing on the fact that near a maximum or minimum, many terms of the series become small and can be ignored. For example, the function is expanded in the neighborhood of \(x_0\), resulting in terms \(n \ln(x_0 + \delta) - (x_0 + \delta)\).
Higher order terms in \(\delta\) (which represents small deviations from \(x_0\)) can be ignored, allowing the expression to be simplified into an approximated form that's easier to integrate, particularly when applied to the specific integral involved in the Stirling approximation.

Integral Evaluation

Evaluating integrals, especially those resulting from approximation, is an essential skill in mathematical methods used for analyzing and approximating functions.
In the context of this exercise, once the original integrand is approximated using logarithmic expansion, the integral is re-evaluated. This requires making certain assumptions, such as extending the lower limit of integration to \(-\infty\), which is valid under the expansion context.
The specific goal in this step is to arrive at the Gaussian integral form which is known to be \(\sqrt{2 \pi} \) when evaluated from \(-\infty\) to \(\infty\). Applying this evaluation leads you to the Stirling's approximation formula for factorial:
\[ n! = n^n e^{-n} (2 \pi n)^{1/2} \]
The factorial, originally a product, transforms into an exponential function with adjustments from the integral evaluation, providing a powerful approximation used in many fields including statistics and physics.