Question

The approximation discussed in Exercise \(3.2 .8\) can be made precise in the following way. Suppose \(X_{n}\) is binomial with the parameters \(n\) and \(p=\lambda / n\), for a given \(\lambda>0 .\) Let \(Y\) be Poisson with mean \(\lambda\). Show that \(P\left(X_{n}=k\right) \rightarrow P(Y=k)\), as \(n \rightarrow \infty\), for an arbitrary but fixed value of \(k\). Hint: First show that: $$ P\left(X_{n}=k\right)=\frac{\lambda^{k}}{k !}\left[\frac{n(n-1) \cdots(n-k+1)}{n^{k}}\left(1-\frac{\lambda}{n}\right)^{-k}\right]\left(1-\frac{\lambda}{n}\right)^{n} $$

Short answer

For \(X_n\) which is a binomial distribution with the parameters \(n\) and \(p=\lambda / n\), it can be proven that when \(n → \infty\) , \(P(X_n=k) → P(Y=k)\) where \(Y= k\) follow Poisson distribution with mean \(λ\). Therefore, it can be concluded that the binomial distribution \(X_n\) converges to a Poisson distribution when \(n\) approaches infinity.

Step-by-step solution

Understand and Write Down the Formulas

Given are the formulas for \(X_n\) and \(Y\) as follows: \(P\left(X_{n}=k\right)=\frac{\lambda^{k}}{k !}\left[\frac{n(n-1) \cdots(n-k+1)}{n^{k}}\left(1-\frac{\lambda}{n}\right)^{-k}\right]\left(1-\frac{\lambda}{n}\right)^{n}\) and \(P(Y=k)=\frac{\lambda^{k}}{k !}\cdot e^{-\lambda}\). The goal is to show \(P\left(X_{n}=k\right) \rightarrow P(Y=k)\) as \(n \rightarrow \infty\).

Rewriting the Formula for \(X_n\)

We want to manipulate the given formula for \(X_n\) to make it more comparable with our Poisson distribution. We can use the properties of exponentiation to simplify the expression as follows: \(P\left(X_{n}=k\right)=\frac{\lambda^{k}}{k !}\left[1\cdot\left(1-\frac{1}{n}\right)\cdots\left(1-\frac{k-1}{n}\right)\right]\left(\left(1-\frac{\lambda}{n}\right)^{-k}\cdot\left(1-\frac{\lambda}{n}\right)^{n}\right)\)

Apply Limit Properties

Now, apply the known limits, that as \(n \rightarrow \infty\), \((1- \lambda/n)^n → e^{-\lambda}\) and \((1- \lambda/n)^{-k} → 1\). Also, the product inside the first brackets approaches 1 as \(n → \infty\). We have \(P\left(X_{n}=k\right)=\frac{\lambda^{k}}{k !}\left(1\right)\left(e^{-\lambda}\right)\) as \(n \rightarrow \infty\)

Compare with Poisson Distribution

Compare this with our definition of the Poisson distribution, \(P(Y=k)=\frac{\lambda^{k}}{k !}\cdot e^{-\lambda}\). They are identical. This proves that the binomial distribution \(X_n\) converges to a Poisson distribution \(Y\) for fixed value of \(k\) when \(n → \infty\). Hence, \(P\left(X_{n}=k\right) \rightarrow P(Y=k)\)

Key concepts

Binomial Distribution

The Binomial Distribution is a discrete probability distribution that models the number of successes in a fixed number of independent Bernoulli trials. A Bernoulli trial is a random experiment where there are only two possible outcomes, commonly termed as a success or a failure. The parameters for a binomial distribution are the number of trials, denoted by n, and the probability of success in each trial, denoted by p.

The probability of getting exactly k successes in n trials is given by the formula:
\[ P(X=k) = {n \choose k}p^k(1-p)^{n-k}\]
where \({n \choose k}\) represents the binomial coefficient, which calculates the number of ways to choose k successes out of n trials. As we explore the behavior of a binomial distribution under certain conditions, we pave the way to understanding its relationship with the Poisson distribution.

Poisson Distribution

The Poisson Distribution is another discrete probability distribution that is used to model the number of events occurring within a fixed interval of time or space, given the events occur with a known constant mean rate and independently of the time since the last event. The distribution is characterized by its mean λ (lambda), which is also its variance.

The formula for the Poisson probability of observing k events is:
\[ P(Y=k) = \frac{λ^k}{k!}e^{-λ}\]
This probability distribution becomes relevant when considering processes that involve rare or random events, like radioactive decay or the number of phone calls received at a call center in an hour. Understanding the nature of how binomial distributions can transition to resembling a Poisson distribution when certain conditions are met will shed light on the concept of probability convergence.

Probability Convergence

Probability Convergence refers to the idea that a sequence of probability distributions approaches a particular distribution as a certain parameter increases without bound. In the context of our discussion, convergence means that as the number of trials n in the binomial distribution becomes very large, and the probability p becomes very small such that the product np (mean number of successes) remains fixed, the binomial distribution approaches the Poisson distribution.

This convergence is particularly useful when dealing with large numbers of trials or rare events, where calculating probabilities using the binomial formula becomes impractical. Therefore, the Poisson distribution can be used as an approximation, simplifying calculations. Convergence is perhaps one of the most practical and insightful phenomena within probability theory, illustrating how different distributions can be interrelated and offering simpler methods for computation.

Limit Theorems in Probability

The Limit Theorems in Probability, such as the Law of Large Numbers and the Central Limit Theorem, are fundamental principles that describe the behavior of probabilities and random variables as we increase the number of observations or trials. In the case of transitioning from a binomial to a Poisson distribution, we're dealing with a specific kind of limit theorem – one that focuses on the convergence of sequences of probability distributions.

This particular application falls under a genre of limit theorems concerning discrete distributions. Here, the limit theorem states that under certain conditions (large n and small p), the distribution of binomial random variables converges to the Poisson distribution. By applying the principles of limit theorems, we can justify the use of the Poisson distribution as a reliable approximation for the binomial distribution when the prescribed conditions are met. Thus, in practical terms, these theorems provide both a deep theoretical understanding and a toolkit for solving real-world problems involving probability.