Question

Let \(X_{1}\) and \(X_{2}\) be independent geometric random variables having the same parameter \(p\). Guess the value of $$ P\left\\{X_{1}=i \mid X_{1}+X_{2}=n\right\\} $$ Hint: Suppose a coin having probability \(p\) of coming up heads is continually flipped. If the second head occurs on flip number \(n\), what is the conditional probability that the first head was on flip number \(i, i=1, \ldots, n-1 ?\) Verify your guess analytically.

Short answer

The short answer to the question is: $$ P\left\{X_1=i \mid X_1+X_2=n\right\} = \frac{1}{n-1} $$ This means that if the second head occurs on the \(n\)-th flip, the probability that the first head occurred on any specific flip between \(1\) and \(n-1\) is uniform, i.e., the events are equally likely, regardless of the value of \(p\).

Step-by-step solution

Identify the Conditional Probability

We are asked to find the probability that the first head, represented by random variable \(X_1\), occurs on the \(i\)-th flip, given that the total number of flips to get the second head is \(n\). In other words, we want to find \(P\left\{X_1=i \mid X_1+X_2=n\right\}\).

Use the Hint

The hint suggests relating the problem to flipping a coin which has a probability \(p\) of showing heads. We are supposed to find out the conditional probability that the first head appears on flip \(i\) given that the second head occurs on flip \(n\).

Relate to Geometric Distribution

In a geometric distribution, the probability that the first head occurs on the \(i\)-th flip is given by \(P(X_1 = i) = (1-p)^{i-1}p\). Similarly, the probability that the second head occurs after \(n-i\) more flips is given by \(P(X_2 = n-i) = (1-p)^{n-i-1}p\). Since \(X_1\) and \(X_2\) are independent, the joint probability of the events is given by the product of their probabilities: \(P(X_1 = i, X_2 = n-i) = (1-p)^{i-1}p(1-p)^{n-i-1}p = (1-p)^{n-2}p^2\).

Apply the Definition of Conditional Probability

To find the conditional probability, we apply the definition: \(P\left\{X_1=i \mid X_1+X_2=n\right\} = \frac{P(X_1=i, X_2=n-i)}{P(X_1+X_2=n)}\). To compute the denominator, we need to sum the joint probabilities over all possible values of \(i\): $$P(X_1+X_2=n) = \sum_{i=1}^{n-1} P(X_1=i, X_2=n-i) = \sum_{i=1}^{n-1} (1-p)^{n-2}p^2 = (n-1)(1-p)^{n-2}p^2$$ Therefore, the conditional probability is: $$P\left\{X_1=i \mid X_1+X_2=n\right\} = \frac{(1-p)^{n-2}p^2}{(n-1)(1-p)^{n-2}p^2} = \frac{1}{n-1}$$

Verification of the Guess

In the previous step, we calculated the conditional probability and arrived at the result \(P\left\{X_1=i \mid X_1+X_2=n\right\} = \frac{1}{n-1}\). This means that regardless of the value of \(p\), if the second head occurs on the \(n\)-th flip, the probability that the first head occurred on any specific flip between \(1\) and \(n-1\) is uniform, i.e., the events are equally likely.

Key concepts

Geometric Random Variables

When we talk about geometric random variables, we're referring to a type of probability distribution that models the number of trials needed to achieve the first success in a series of independent Bernoulli trials. A Bernoulli trial is an experiment that has exactly two outcomes: success or failure. For instance, when flipping a coin, getting a 'head' could be considered a 'success,' given that the probability of heads is constant with every flip.

Mathematically, if the probability of success on a single trial is denoted by p, the probability that the first success occurs on the i-th trial is represented as:
\( P(X = i) = (1 - p)^{i - 1}p \).

This formula implies that the previous i - 1 trials were failures (hence, the (1 - p) term raised to the power of i - 1), followed by a success on the i-th trial (indicated by the p term). Geometric random variables are memoryless, meaning that the probability of success is the same regardless of any previous failures.

Independent Random Variables

The concept of independent random variables is core to many probability problems. Two random variables, X and Y, are considered independent if the occurrence of one does not affect the probability of the occurrence of the other. This means knowing the outcome of X provides no information about Y, and vice versa.

For independent random variables, the joint probability of two events happening (for example, X = x and Y = y) is the product of their individual probabilities:\( P(X = x \text{ and } Y = y) = P(X = x) \cdot P(Y = y) \).

In the context of the exercise, the independence between X₁ and X₂, the number of trials before the first and second success, is crucial. It ensures that the number of flips to get the first head is not affected by the number of flips that follow before getting the second head.

Probability Distribution

Probability distribution is a term that describes how the probabilities of a random variable are distributed over all of its possible values. It provides the likelihood of each outcome and is fundamental to understanding various phenomena in probability theory. Each random variable has its own distribution, which might be uniform, binomial, normal, or another type depending on the context.

Probability distribution comes in two forms: discrete and continuous. Discrete distributions apply to scenarios with countable outcomes, like flipping coins or rolling dice. On the other hand, continuous distributions apply when the outcomes are points on a continuum, such as measurements of height or weight.

In our exercise, we deal with a discrete probability distribution since the outcome (number of flips before the first or second heads) is countable. Understanding the distribution helps us calculate probabilities for specific events, like finding the exact number of trials before a success or a range of outcomes.

Coin Flipping Probability

Interestingly, the simple act of coin flipping provides a familiar model for introducing various probability concepts including geometric distribution and independence. Each flip of a fair coin is an independent event with a probability distribution where the likelihood of a 'head' or a 'tail' is equal — each having a probability of 0.5 (assuming the coin is fair).

However, not all coins are fair, and some might have a different probability p of landing on heads. When we are dealing with weighted coins in our problems, the coin flipping probability generalizes to:
\( P(\text{head}) = p \) and \( P(\text{tail}) = 1 - p \).

The power of the coin flipping model lies in its simplicity for visualizing random processes and outcomes. As seen in the exercise, we can model complex situations, such as finding the conditional probability of the first head occurring on a specific flip, by applying principles derived from a fundamental understanding of coin flipping probability.