Question

Let \(X_{1}, X_{2}, \ldots, X_{n}\) be random sample from the geometric distribution with PMF $$ p(x ; q)=\left\\{\begin{array}{l} q^{x}(1-q), \quad x=0,1,2, \ldots ; 0 \leq q \leq 1 \\ 0 \quad \text { otherwise } \end{array}\right. $$ 1\. Find the \(\mathrm{MLE} \hat{q}\) of \(q\). 2\. Show that \(\sum_{i=1}^{n} X_{i}\) is a complete sufficient statistics for \(q\). 3\. Determine the minimum variance unbiased estimator of \(q\).

Short answer

The Maximum Likelihood Estimator (MLE) of 'q' in the given geometric distribution is found by deriving and setting the derivative of the log-likelihood function equal to zero. The statistic \(\sum_{i=1}^{n} X_{i}\) is a complete and sufficient statistic for 'q' according to the factorization theorem. The minimum variance unbiased estimator of 'q' is given by \(q = \frac{1}{{\bar{X}}+1}\).

Step-by-step solution

Find the Maximum Likelihood Estimator

A geometric distribution has probability mass function \(p(x ; q) = q^{x}(1-q)\), for \(x=0,1,2, \ldots\) and \(0 \leq q \leq 1\). We can derive the likelihood function of our sample: \[L(q) = \prod_{i=1}^n p(x_i;q) = \prod_{i=1}^n q^{x_i}(1-q)\]. This simplifies to \[L(q) = q^{\sum_{i=1}^n x_i} (1-q)^n\]. Taking the logarithm of our likelihood function gives us \[ll(q) = ln(L(q)) = \sum_{i=1}^n x_i ln(q) + n ln(1-q)\]. Taking the derivative of the log-likelihood function with respect to 'q' and setting it equal to zero, we get: \(0 = \sum_{i=1}^n \frac{x_i}{q} - \frac{n}{1-q}\). Solving this equation for 'q' will yield the MLE of \(q\), \(\hat{q}\).

Show the completeness and sufficiency of statistics

To examine the completeness and sufficiency of the statistic \(\sum_{i=1}^{n} X_{i}\), we can use the factorization theorem. We can rewrite the joint PMF as \[p(x ; q) = q^{\sum_{i=1}^{n} x_i} (1-q)^n = q^T (1-q)^n\] where \(T = \sum_{i=1}^{n} X_{i}\). It has been factorized into two parts, one only depends on the sample data through the statistic \(T\), and the other does not depend on the data. Which according to the factorization theorem, \(T\) is a sufficient statistic for 'q'. Since there's only one sufficient statistic, we can say \(T\) is a complete statistic for 'q'.

Find the minimum variance unbiased estimator

An unbiased estimator for 'q' is \(q = \frac{1}{{\bar{X}}+1}\), where \(\bar{X}\) is the sample mean. For the given geometric distribution, the variance of 'q' is \(Var[q] = \frac{(1-q)^2}{(nq^2)}\). By the Cramer-Rao inequality, this estimator has the minimum variance among all unbiased estimators of 'q'. So the minimum variance unbiased estimator of 'q' is \(q = \frac{1}{{\bar{X}}+1}\).

Key concepts

Maximum Likelihood Estimation

When working with statistical models, one of the fundamental methods for estimating the parameters of a distribution is Maximum Likelihood Estimation (MLE). It involves choosing parameter values that maximize the likelihood of the observed data under the statistical model. In simpler terms, it identifies which parameter values are most likely to produce the data that we have observed.

For a geometric distribution with a probability mass function (PMF) given by \( p(x ; q) = q^{x}(1-q) \), where \(q\) is the probability of success, the likelihood function is the product of the individual probabilities for all observations in the sample. In an MLE approach, to find the best estimate of \(q\), denoted by \(\hat{q}\), we calculate the likelihood function \(L(q)\) for our sample and then maximize it. Typically, the logarithm of the likelihood function, called the log-likelihood, is easier to maximize because it turns products into sums, making the differentiation manageable.

To find the MLE, \(\hat{q}\), we differentiate the log-likelihood function with respect to \(q\) and set the derivative equal to zero. Finding the value of \(q\) that satisfies this, gives us the MLE, which is the value that makes the observed data most probable. The MLE utilizes the data efficiently and is favored for its good asymptotic properties, such as consistency and efficiency under regular conditions.

Complete Sufficient Statistics

Within the realm of statistics, a statistic is said to be sufficient for a parameter if no other statistic can provide any additional information as to the value of the parameter. The formal definition is provided by the factorization theorem, which states that a statistic \(T\) is sufficient for a parameter \(q\) if the joint probability distribution of the sample \(X_{1}, X_{2}, \text{...}, X_{n}\) can be expressed as a product of two functions: one that depends only on the sample data through \(T\) and the other that depends only on the parameter \(q\).

In the context of a geometric distribution, we consider the statistic \(\sum_{i=1}^{n} X_{i}\), which is the sum of all the random variables in the sample. This statistic captures all the information available about the parameter \(q\), thus qualifying as a sufficient statistic for \(q\). Moreover, a statistic is complete if it contains all the information about the parameter that the sample can provide. Completeness implies that there are no other statistics that, given the value of the complete statistic, can provide additional information about the parameter. We can determine that \(\sum_{i=1}^{n} X_{i}\) is complete by ensuring that for every measurable function \(g\), if the expected value of \(g(T)\) is zero for all values of \(q\), then \(g(T)\) must be zero with probability one. This property of completeness is highly desirable as it ensures that the sufficient statistic has captured all the information, which in turn prevents contradictory estimators of \(q\).

Minimum Variance Unbiased Estimator

Estimators play a crucial role in using sample data to make inferences about a population parameter. An unbiased estimator is one where, on average, the estimate is equal to the true parameter value. This means that it does not systematically overestimate or underestimate the parameter. However, among the unbiased estimators, some have lower variance than others, meaning their estimates are more consistent across different samples.

The Minimum Variance Unbiased Estimator (MVUE) for a parameter is the one with the least variance among all unbiased estimators for that parameter, essentially providing the most precise estimates. For the geometric distribution, an unbiased estimator for \(q\) is \(\frac{1}{\bar{X}+1}\), with \(\bar{X}\) being the sample mean. To affirm this estimator is indeed an MVUE, it must satisfy the Cramér-Rao lower bound which provides a theoretical lower limit to the variance that any unbiased estimator of a parameter can have.

Technically, for geometric distribution, the variance of this unbiased estimator is minimized according to the relation \(Var[q] = \frac{(1-q)^2}{(nq^2)}\). Since this relationship adheres to the Cramér-Rao inequality, it implies the estimator achieves minimum variance. Confirming \(\frac{1}{\bar{X}+1}\) as the MVUE ensures the most accurate and reliable estimation of \(q\) given the constraints of the geometric distribution model and our sample data.