Question

Recall that by definition, a discrete sample space may contain a countably infinite number of outcomes. This exercise gives an example of such a countably infinite sample space. Suppose we flip a fair coin until it comes up heads. Of course, there is no way to know in advance how many flips will be required. Design a sample space and a probability density to model this situation. Prove that the probability density you define is legitimate.

Short answer

The sample space is \( \{ H, TH, TTH, \ldots \} \) and the probability density is \( P(X=n) = \left( \frac{1}{2} \right)^n \) for \( n \geq 1 \), which sums to 1.

Step-by-step solution

Define the Sample Space

The sample space \( S \) for this scenario is the set of all possible sequences of coin flips where the first head appears. This can be represented as \( S = \{ H, TH, TTH, TTTH, \ldots \} \). Each outcome corresponds to a sequence where a tail (T) is followed by another tail, and so on, until the first head (H) appears. Thus, each outcome is identified by the number of trials it took to achieve the first head.

Define the Probability Density

The probability of achieving the first head on the \( n \)-th flip is \( P(X = n) \). For a fair coin, the probability of a tail followed by the first head at the \( n \)-th flip is \( \left( \frac{1}{2} \right)^{n-1} \times \frac{1}{2} = \left( \frac{1}{2} \right)^n \). Hence, \( P(X = n) = \left( \frac{1}{2} \right)^n \) for \( n = 1, 2, 3, \ldots \).

Verify the Probability Distribution

To verify this is a legitimate probability distribution, we must show that the probabilities sum to 1. We calculate the sum: \[ \sum_{n=1}^{\infty} P(X = n) = \sum_{n=1}^{\infty} \left( \frac{1}{2} \right)^n = \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \cdots \] This is an infinite geometric series with first term \( a = \frac{1}{2} \) and ratio \( r = \frac{1}{2} \). The sum of a geometric series is given by \( \frac{a}{1-r} \), which in this case is: \[ \frac{\frac{1}{2}}{1 - \frac{1}{2}} = 1 \] Thus, the probabilities sum to 1, confirming the distribution is valid.

Key concepts

Probability Density

When dealing with discrete sample spaces, we often need to define a probability density to assign probabilities to each possible outcome. In the context of flipping a fair coin until a head appears, the probability density function helps us determine the likelihood of achieving this outcome on a specific flip.
For the given problem, the probability of getting the first head on the \( n \)-th flip is represented by \( P(X = n) = \left( \frac{1}{2} \right)^n \). This formula is derived from multiplying the probability of getting consecutive tails (\( \frac{1}{2} \) for each tail) followed by a head (\( \frac{1}{2} \)).
Such a function must satisfy two conditions:

• Each probability value must be between 0 and 1.
• The sum of all probabilities must equal 1.

This ensures that the probability distribution is accurate and complete.

Geometric Series

A geometric series is a series of terms that have a constant ratio between successive terms. In our problem, this series represents the total probability of getting heads at any time during the sequence of coin flips.
The probability for the first head to occur is expressed as an infinite geometric series: \( \sum_{n=1}^{\infty} \left( \frac{1}{2} \right)^n = \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \ldots \). This series has a first term \( a = \frac{1}{2} \) and a common ratio \( r = \frac{1}{2} \).
The formula for the sum of a geometric series \( S \) is given by \( \frac{a}{1-r} \). In this case, substituting the values, we find \( \frac{\frac{1}{2}}{1 - \frac{1}{2}} = 1 \). This confirms that the series sums to 1, validating the entire probability distribution.

Fair Coin

A fair coin is a theoretical concept where the coin has an equal likelihood of landing on either heads or tails. This means the probability of getting a head is \( \frac{1}{2} \), as is the probability of getting a tail, making it unbiased.
In our exercise, assuming a fair coin model allows us to predict each flip's outcome with this simple probability, simplifying the probability density calculations.
With the sequence of events (like getting multiple tails before a head), the fair nature of the coin leads to multiplying \( \frac{1}{2} \) for every additional flip. Understanding a fair coin's principle is crucial because it forms the foundation of calculating probabilities in sequential experiments like the coin flipping exercise in question.