Question

Suppose \(X_{1}, X_{2}, \ldots, X_{n}\) are iid with pdf \(f(x ; \theta)=(1 / \theta) e^{-x / \theta}, 0

Short answer

The maximum likelihood estimator of \(P(X \leq 2)\) is \(P_{MLE}(X \leq 2) = 1 - e^{-\frac{2n}{\sum_{i=1}^n x_i}}\). It has been demonstrated to be consistent as the sample size tends to infinity.

Step-by-step solution

Formulate The Likelihood Function

Given iid random variables with probability density function \(f(x ; \theta)=(1 / \theta) e^{-x / \theta}, 0 < x < \infty\), the joint probability function is the product of individual densities. Therefore, the likelihood function \(L(\theta)\) is:\(L(\theta) = \prod_{i=1}^n f(x_i; \theta) = \prod_{i=1}^n (1 / \theta) e^{-x_i / \theta} = (1/\theta^n) e^{-\frac{1}{\theta}\sum_{i=1}^n x_i}\).

Maximising The Log-Likelihood Function

To find the MLE, we maximise the likelihood function. Given that the natural logarithm is a strictly increasing function, that can be done by maximising the log-likelihood function instead. The log-likelihood function \(l(\theta)\) is:\(l(\theta) = log(L(\theta)) = -n log(\theta) - \frac{1}{\theta}\sum_{i=1}^n x_i\). Differentiating \(l(\theta)\) with respect to \(\theta\) and setting it to zero, we get:\(-n/\theta + \sum_{i=1}^n x_i / \theta^2 = 0 \). Solving for \(\theta\) gives the maximum likelihood estimator \(\theta_{MLE} = \sum_{i=1}^n x_i / n\).

Estimate Cumulative Distribution Function P(X ≤ 2)

Having found the \(\theta_{MLE}\), the cumulative distribution function \(P(X \leq 2)\) can be calculated as \(1 - e^{-\frac{2}{\theta_{MLE}}}\) using properties of exponential distribution. Inserting \(\theta_{MLE} = \sum_{i=1}^n x_i / n\), we have:\(P_{MLE}(X \leq 2) = 1 - e^{-\frac{2n}{\sum_{i=1}^n x_i}}\).

Demonstrate The Consistency Of The Estimator

To show that \(P_{MLE}(X \leq 2)\) is a consistent estimator of \(P(X \leq 2)\), we need to prove that the expectation of the estimator equals the parameter being estimated and that its variance goes to zero as the sample size tends to infinity:1. Given the Weak Law of Large Numbers, as \(n\) approaches infinity, the mean \(\sum_{i=1}^n x_i / n\) converges in probability to \(\theta\), the expected value of the population. Hence, \(\theta_{MLE}\) is an unbiased estimator of \(\theta\).2. Also, the Central Limit Theorem assures that as the sample size increases, the variance of the sample mean decreases. By substitution, it's easily seen that the variance of \(P_{MLE}(X \leq 2)\) also decreases as \(n\) increases.Thus, \(P_{MLE}(X \leq 2)\) is a consistent estimator of \(P(X \leq 2)\).

Key concepts

Probability Density Function

The probability density function (PDF) is a fundamental concept in statistics used to describe the likelihood of a continuous random variable taking on a certain value. The PDF is denoted as a function, like the given exercise example \( f(x; \theta) = (1 / \theta)e^{-x / \theta} \) for \( 0 < x < \infty \) and zero elsewhere. This particular function represents an exponential distribution, characterized by the parameter \( \theta \).

The PDF serves to provide a formula for the probability of the variable falling within a particular range. It's important to note that, while the PDF can give us the probability density at a specific point, we cannot directly obtain the probability of the exact value of a continuous random variable. Instead, we use integrals over the PDF to find out the probability of the variable lying in a certain interval. This is why for continuous random variables, the probability at a single point is always zero, and we rely on the cumulative distribution function (CDF) to provide probabilities over intervals.

Cumulative Distribution Function

The cumulative distribution function (CDF), on the other hand, is a measure that yields the probability that a random variable will be less than or equal to a particular value. Unlike the PDF, the CDF gives a cumulative probability and is defined for all values that a random variable can take on. In the context of the maximum likelihood estimation (MLE) example from the exercise, the CDF of \( X \) is calculated as \( P(X \leq x) \), which equates to \( 1 - e^{-x / \theta} \) for the exponential distribution.

The exercise's solution uses the property of exponential distribution to find the likelihood that \( X \leq 2 \) by substituting the MLE of \( \theta \) into this formula. As we can see, the connection between PDF and CDF is intimate, where the CDF is the integral of the PDF from negative infinity to \( x \) and equivalently, the PDF is the derivative of the CDF.

Consistency of Estimator

Consistency of an estimator is a property that assures us that as the sample size goes to infinity, the estimator converges in probability to the true value of the parameter being estimated. This concept is a cornerstone for establishing the reliability of an estimator in inferential statistics. In the given solution, consistency is demonstrated through the use of two important theorems: the Weak Law of Large Numbers and the Central Limit Theorem.

The Law of Large Numbers tells us that, with a larger sample, the sample mean will get closer to the expected value, making our MLE of \( \theta \) more accurate. Furthermore, the Central Limit Theorem implies the variance of the estimator decreases as the sample size \( n \) increases, reinforcing that the estimator becomes more precise. Therefore, the exercise solution shows that \( P_{MLE}(X \leq 2) \) is consistent because it meets these conditions, reinforcing its reliability for large sample sizes.