Question

. Let the random variable \(X\) be \(N\left(\mu, \sigma^{2}\right) .\) What would this distribution be if \(\sigma^{2}=0 ?\) Hint: Look at the mgf of \(X\) for \(\sigma^{2}>0\) and investigate its limit as \(\sigma^{2} \rightarrow 0\).

Short answer

When \(\sigma^{2}=0\), the distribution of \(X\) will be a degenerate distribution at \(\mu\).

Step-by-step solution

Moment Generating Function (MGF) of Normal Distribution

The mgf of a normal distribution \(N\left(\mu, \sigma^{2}\right)\) is given by \(M(t) = e^{\mu t + 0.5\sigma^{2}t^{2}}\).

Set \(\sigma^{2}=0\)

Substitute the value of \(\sigma^{2}\) with 0 in the mgf of the normal distribution. Then, it becomes \(M(t) = e^{\mu t + 0.5*0*t^{2}} = e^{\mu t}\).

Identify the Probability Distribution

The resultant mgf, \(e^{\mu t}\), is the moment generating function of a degenerate random variable at a point \(\mu\). Thus, when \(\sigma^{2}=0\), the normal distribution \(N\left(\mu, \sigma^{2}\right)\) will become a degenerate distribution at \(\mu\).

Key concepts

Moment Generating Function

The moment generating function (MGF) is a very powerful tool in probability theory that provides a unique characterization of a probability distribution. It essentially encodes all the moments (i.e., expected values) of a random variable into a single function. For any random variable, the MGF is defined as \( M(t) = E(e^{tX}) \), where \( E \) represents the expected value, \( X \) is the random variable in question, and \( t \) is a real number for which the expected value exists.

For a normal distribution, which is defined by its mean \( \mu \) and variance \( \sigma^2 \), the MGF is given by the formula \( M(t) = e^{\mu t + \frac{1}{2}\sigma^2t^2} \). When analyzing the behavior of the MGF, and subsequently the behavior of the distribution itself, changing the variance can lead to interesting insights into the nature of the distribution, especially as we look at the limiting behavior where the variance approaches zero.

Degenerate Distribution

A degenerate distribution is a special kind of probability distribution in which the random variable is almost surely constant. In simpler terms, this means that the variable takes on a single value with a probability of one.

This concept may appear strange in the context of probability theory since we often think about random variables as inherently uncertain. However, degenerate distributions serve as important edge cases or baselines. They occur when there's no randomness in the behavior of the variable - it's fully determined to be a specific value.

Considering the moment generating function of a degenerate distribution concentrated at a point \( \mu \), it is merely \( M(t) = e^{\mu t} \). This function aligns with what we would intuitively expect: it represents all moments of a random variable that does not vary at all. The distributions with more variance spread out the probability mass over a range of values, but the degenerate distribution collapses all that mass into a single point.

Limit of MGF as Variance Approaches Zero

Investigating the limit of the moment generating function as the variance approaches zero bridges the gap between the normal and degenerate distributions. Remember, the MGF for a normal distribution is given by \( M(t) = e^{\mu t + \frac{1}{2}\sigma^2t^2} \). As \( \sigma^2 \) approaches zero, the quadratic term in the exponent becomes insignificant, and the MGF simplifies to \( M(t) = e^{\mu t} \), which, as previously mentioned, is the MGF of a degenerate distribution.

This convergence reveals that a normal distribution with mean \( \mu \) and an infinitesimally small variance behaves like a random variable that is essentially equal to \( \mu \) with certainty. Conceptually, this represents a situation where there is no 'spread' or uncertainty around the mean - which is the very essence of a degenerate distribution. By exploring the behavior of the MGF in these limits, one can gain a deeper understanding of the relationship between variance and distribution shape.