The Stirling approximation is useful in a variety of different settings. The
goal of the present problem is to work through a more sophisticated treatment
of this approximation than the simple heuristic argument given in the chapter.
Our task is to find useful representations of \(n !\) since terms of the form In
\(n !\) arise often in reasoning about entropy.
(a) Begin by showing that
$$n !=\int_{0}^{\infty} x^{n} e^{-x} d x$$
To demonstrate this, use repeated integration by parts. In particular,
demonstrate the recurrence relation
$$\int_{0}^{\infty} x^{n} \mathrm{e}^{-x} \mathrm{d} x=n \int_{0}^{\infty}
x^{n-1} \mathrm{e}^{-x} \mathrm{d} x$$
and then argue that repeated application of this relation leads to the desired
result.
(b) Make plots of the integrand \(x^{n} \mathrm{e}^{-x}\) for various values of
\(n\) and observe the peak width and height of this integrand. We are interested
now in finding the value of \(x\) for which this function is a maximum. The idea
is that we will then expand about that maximum. To carry out this step,
consider \(\ln \left(x^{n} \mathrm{e}^{-x}\right)\) and find its maximum-argue
why it is acceptable to use the logarithm of the original function as a
surrogate for the function itself, that is, show that the maxima of both the
function and its logarithm are at the same \(x\), Also, argue why it might be a
good idea to use the logarithm of the integrand rather than the integrand
itself as the basis of our analysis. Call the value of \(x\) for which this
function is maximized \(x_{0}\). Now expand the logarithm about \(x_{0}\). In
particular, examine
\\[
\ln \left[\left(x_{0}+\delta\right)^{n}
\mathrm{e}^{-\left(x_{0}+\delta\right)}\right]=n \ln
\left(x_{0}+\delta\right)-\left(x_{0}+\delta\right)
\\]
and expand to second order in \(8 .\) Exponentiate your result and you should
now have an approximation to the original integrand that is good in the
neighborhood of \(x_{0}\). Plug this back into the integral (be careful with
limits of integration) and, by showing that it is acceptable to send the lower
limit of integration to \(-\infty,\) show that
\\[
n ! \approx n^{n} \mathrm{e}^{-n} \int_{-\infty}^{\infty}
\mathrm{e}^{-\delta^{2} / 2 n} \mathrm{d} \delta
\\]
Evaluate the integral and show that in this approximation
\\[
n !=n^{n} e^{-n}(2 \pi n)^{1 / 2}
\\]
Also, take the logarithm of this result and make an argument as to why most of
the time we can get away with dropping the \((2 \pi n)^{1 / 2}\) term.