Question

Let \(X_{1}, X_{2}, \ldots, X_{n}\) be a random sample from the modified geometric distribution with PMF $$ p(x ; q)=(1-q)^{x-1} q, \quad x=1,2, \ldots $$ Prove that maximum likelihood estimator of \(q\) is $$ \hat{q}=\frac{n}{\sum_{i=1}^{n} x_{i}}=\frac{1}{\bar{x}} $$

Short answer

The maximum likelihood estimator of \(q\) is \(\hat{q} = \frac{1}{\bar{x}}\).

Step-by-step solution

Setting Up the Likelihood Function

Given the PMF \( p(x ; q)=(1-q)^{x-1} q \), for a sample \( x_1, x_2, \ldots, x_n \), the likelihood function \( L(q) \) is formed by evaluating the PMF at the sample points and taking the product over all sample points: \[ L(q) = \prod_{i=1}^{n} (1-q)^{x_i-1} q \]It can also be written as: \[ L(q) = q^n (1-q)^{\sum_{i=1}^n (x_i - 1)} \]

Calculating the Log-Likelihood Function

The logarithm simplifies our product to a sum, making derivative calculation easier. Take the logarithm (natural log here) on both sides of the equation:\[ \ln(L(q)) = \ln\left( q^n (1-q)^{\sum_{i=1}^n (x_i - 1)} \right) = n \ln(q) + \left(\sum_{i=1}^n (x_i - 1)\right) \ln(1-q) \]

Differentiating the Log-Likelihood and Setting it to Zero

To find the maximum of the function, take the derivative and set it equal to zero. Differentiating \(\ln(L(q))\) with respect to \(q\), that yields:\[ \frac{d \ln(L(q))}{dq} = \frac{n}{q} - \frac{\sum_{i=1}^{n} (x_i - 1)}{1-q} = 0 \]

Solving the Resulting Equation for \(q\)

Solving the equation for \(q\), we get:\[ n - q\sum_{i=1}^{n} (x_i - 1) = 0 \implies q = \frac{n}{\sum_{i=1}^{n} (x_i)} = \frac{1}{\bar{x}} \]where \(\bar{x}\) is the sample mean.

Key concepts

Geometric Distribution

The geometric distribution is a fascinating discrete probability distribution. It represents the number of trials needed to get the first success in a sequence of independent and identical trials. Each trial is characterized by two possible outcomes: success or failure. The probability of success in each trial is denoted by a parameter, typically called \( q \).

In the geometric distribution, each trial is independent, meaning the outcome of one trial doesn't affect the others. The probability mass function (PMF) of the geometric distribution is given by \( p(x; q) = (1-q)^{x-1} q \), where \( x \) is the number of trials. Here, the parameter \( q \) is the probability of success, and \((1-q)\) is the probability of failure. The exponent \( x-1 \) reflects the number of failures before the first success.

The geometric distribution is helpful in modeling scenarios like flipping a coin until heads appears or rolling a dice until a six is rolled. Understanding this distribution is crucial for deriving estimators such as the maximum likelihood estimator, as it helps in understanding how the data is statistically distributed.

Log-Likelihood Function

In statistics, the log-likelihood function is an essential concept used to find parameters that best fit a given statistical model. When dealing with likelihood functions, taking the logarithm simplifies the product of probabilities into a sum, which is typically easier to handle mathematically. This transformation is quite useful in maximizing or optimizing these functions.

For the given geometric distribution with PMF \( p(x; q) = (1-q)^{x-1} q \), the likelihood function is created by evaluating this PMF for a set of independent observations and then multiplying them together. It is expressed as:

• \( L(q) = \prod_{i=1}^{n} (1-q)^{x_i-1} q \)

Applying log transformation, the log-likelihood function becomes:

• \( \ln(L(q)) = n \ln(q) + \left( \sum_{i=1}^n (x_i - 1) \right) \ln(1-q) \)

This expression is a sum which makes it easier to compute derivatives (needed for finding maxima or minima in calculus). By calculating the derivative of the log-likelihood function and setting it to zero, we can find the values of \( q \) that maximize the likelihood of the observed data.

Random Sample

The concept of a random sample is a cornerstone in statistical analysis and inference. A random sample consists of independent observations drawn from a larger population where each individual observation is equally likely to be selected. Such sampling enables the derivation of insights about the entire population based on the sample data.

In the context of the geometric distribution, when a sample \( X_1, X_2, \ldots, X_n \) is mentioned, it refers to these independent observations drawn from the geometric distribution. Each \( X_i \) represents an outcome following the geometric properties with a certain probability \( q \) of occurrence.

Random sampling ensures that calculations, like determining the maximum likelihood estimator, yield results valid for the entire population. This unbiased sampling is critical when estimating parameters such as \( q \) in the PMF of the geometric distribution. By employing statistical methods like maximum likelihood estimation on random samples, meaningful inferences about the population parameters can be drawn, aiding in decision-making processes.