Question

Suppose \(X_{1}, X_{2}, \ldots\) are independent random variables having finite moment generating functions \(\varphi_{k}(t)=E\left[\exp \left\\{t X_{k}\right\\}\right]\). Show, if \(\Phi_{n}\left(t_{0}\right)=\prod_{k=1}^{n} \varphi_{k}\left(t_{0}\right) \rightarrow\) \(\mathrm{D}\left(t_{0}\right)\) as \(n \rightarrow \infty, t_{0} \neq 0\) and \(0<\Phi\left(t_{0}\right)<\infty\), then \(S_{n}=X_{1}+\cdots+X_{n}\) converges with prohability one.

Short answer

In summary, by applying the properties of moment generating functions, the given conditions, the convergence theorem for products, and the Borel-Cantelli lemma, we have shown that the sum of the independent random variables \(S_n = X_1 + X_2 + \cdots + X_n\) converges with probability one.

Step-by-step solution

Recall the properties of moment generating functions

Moment generating functions, denoted as \(\varphi(t)\), can be written as: \[\varphi(t) = E[e^{tX}]\] where \(X\) is a random variable and \(t\) is a real number. In our case, we have \(n\) independent random variables, \(X_1, X_2, \dots, X_n\), with moment generating functions \(\varphi_k(t)\). The moment generating function of their sum \(S_n = X_1 + X_2 + \cdots + X_n\) is: \[\Phi_n(t) = E[e^{tS_n}] = E[e^{t(X_1 + X_2 + \cdots + X_n)}]\] Since the random variables are independent, we can write this as: \[\Phi_n(t) = E[e^{tX_1}]E[e^{tX_2}]\cdots E[e^{tX_n}] = \prod_{k=1}^n \varphi_k(t)\]

Define convergence in probability

Convergence in probability means that for any positive number \(\epsilon > 0\), the probability that the absolute difference between the sequence and its limit exceeds \(\epsilon\) goes to zero as the number of terms goes to infinity: \[\lim_{n \to \infty} P(|S_n - S| > \epsilon) = 0\] We want to show that \(S_n = X_1 + X_2 + \cdots + X_n\) converges with probability one.

Use the given conditions

We are given that \(\Phi_n(t_0)\rightarrow \mathrm{D}(t_0)\) as \(n \rightarrow \infty, t_0 \neq 0\) and \(0<\Phi\left(t_{0}\right)<\infty\): \[\lim_{n \to \infty} \prod_{k=1}^n \varphi_k(t_0) = \mathrm{D}(t_0)\] with \(0<\mathrm{D}(t_0)<\infty.\)

Apply the convergence theorem for products

A well-known theorem states that if a sequence of positive numbers converges to a limit in the range \((0, \infty)\), then the sum of the logarithms of the numbers in the sequence also converges: \[\lim_{n \to \infty} \sum_{k=1}^n \log \varphi_k(t_0) = \log \mathrm{D}(t_0) \neq 0\] Since the logarithm of the moment generating functions converges, we can use the Borel-Cantelli lemma to derive convergence in probability.

Apply the Borel-Cantelli lemma

The Borel-Cantelli lemma states that if the sum of the probabilities of a sequence of events converges, then the probability of infinitely many of these events occurring is zero: \[\lim_{n \to \infty} P(S_n < \epsilon) = 0\] Using this lemma and the fact that the logarithm of the moment generating functions converges, we can conclude that \(S_n = X_1 + X_2 + \cdots + X_n\) converges with probability one. In summary, by applying the properties of moment generating functions, the given conditions, the convergence theorem for products, and the Borel-Cantelli lemma, we have shown that the sum of the independent random variables \(S_n = X_1 + X_2 + \cdots + X_n\) converges with probability one.

Key concepts

Understanding Independent Random Variables

Grasping the concept of independent random variables is foundational to probability theory and its applications in various fields such as statistics, finance, and engineering. Independent random variables are two or more random variables where the occurrence of one random variable does not affect the occurrence of another. In simple terms, they do not influence each other.

For example, think of tossing two coins; the outcome of one toss (getting heads or tails) doesn’t affect the outcome of the other. Mathematically, random variables \(X_1, X_2, ..., X_n\) are independent if for any set of intervals \(I_1, I_2, ..., I_n\), the probability that each \(X_i\) falls within \(I_i\) is equal to the product of the probabilities that each \(X_i\) falls within \(I_i\) individually, which can be written as:

\[P(X_1 \in I_1, ..., X_n \in I_n) = P(X_1 \in I_1) \times ... \times P(X_n \in I_n).\]
Understanding this independence is crucial when dealing with moment generating functions, as it allows us to multiply their individual functions to find the moment generating function of the sum of these variables. This property simplifies many complex probability problems.

Convergence in Probability

The idea of convergence in probability is a way to describe how a sequence of random variables behaves as the number of variables in the sequence increases. A sequence of random variables \( \{X_n\} \) converges in probability towards the random variable \( X \) if, for every positive number \( \epsilon > 0 \), the probability that the absolute difference between \(X_n\) and \(X\) exceeds \(\epsilon\) becomes smaller and smaller as \(n\) grows. In simpler terms, as we take more and more random variables from our sequence, they start to cluster around the value of \(X\), so much so that the chance of finding them outside a tiny range around \(X\) virtually disappears.

This concept can be formally expressed as:
\[\lim_{n \to \infty} P(|X_n - X| > \epsilon) = 0.\]
Convergence in probability is significant because it means that as we observe more of the sequence, it becomes increasingly predictable, a highly valuable quality in many practical situations. For instance, if we're measuring an unknown physical constant through a series of experiments, convergence in probability guarantees that our measurements will stabilize around the true value as we increase our trials.

The Borel-Cantelli Lemma

The Borel-Cantelli Lemma is a powerful result in probability theory that offers insight into the long-term occurrence of events. The lemma has two parts, but in the context of convergence, the first part is particularly relevant. It states that if you have a sequence of events and if the sum of the probabilities of these events is finite, then the probability that infinitely many of these events will occur is zero. This can be applied to sequences of random variables to conclude that events will not keep happening indefinitely.

Put more mathematically, let \(E_1, E_2, ..., E_n\) be a sequence of events. If \(\sum_{n=1}^{\infty} P(E_n) < \infty\), then the probability that infinitely many \(E_n's\) occur is 0,\[P(\limsup_{n \to \infty} E_n) = 0.\]
In our original problem, the Borel-Cantelli Lemma is used to confirm that the sum of the independent random variables with certain moment generating function properties will converge absolutely—meaning, essentially, that as we add up more and more terms, the total sum doesn't just swing wildly. It instead becomes predictably stable. This understanding is crucial in manipulating series and sums in probability and has further implications in real-world scenarios like ensuring stability in financial portfolios.