Question

(a) Let \(Y_{1}, \ldots, Y_{n}\) be a random sample from the exponential density \(\lambda e^{-\lambda y}, y>0, \lambda>0\) Say why an unbiased estimator \(W\) for \(\lambda\) should have form \(a / S\), and hence find \(a\). Find the Fisher information for \(\lambda\) and show that \(\mathrm{E}\left(W^{2}\right)=(n-1) \lambda^{2} /(n-2)\). Deduce that no unbiased estimator of \(\lambda\) attains the Cramér-Rao lower bound, although \(W\) does so asymptotically. (b) Let \(\psi=\operatorname{Pr}(Y>a)=e^{-\lambda a}\), for some constant \(a\). Show that $$ I\left(Y_{1}>a\right)= \begin{cases}1, & Y_{1}>a \\ 0, & \text { otherwise }\end{cases} $$ is an unbiased estimator of \(\psi\), and hence obtain the minimum variance unbiased estimator. Does this attain the Cramér-Rao lower bound for \(\psi\) ?

Short answer

(a) The unbiased estimator is \(W = \frac{n-1}{S}\); it does not achieve CRLB but is asymptotically efficient. (b) \(I(Y_1 > a)\) is an unbiased estimator for \(\psi\). It does not attain CRLB for \(\psi\).

Step-by-step solution

Understanding the Unbiased Estimator Form

For the exponential distribution, the sum of the sample data, i.e., \( S = \sum_{i=1}^{n} Y_{i} \), follows a gamma distribution with parameters \( n \) and \( \lambda \) (i.e., \( S \sim \text{Gamma}(n,\lambda) \)). An unbiased estimator \( W \) of \( \lambda \) must have expected value equal to \( \lambda \). Since \( S \) is a sufficient statistic for \( \lambda \), we consider an estimator of the form \( W = \frac{a}{S} \). This requires \( \mathbb{E}\left(\frac{a}{S}\right) = \lambda \) to hold.

Calculating the Constant 'a'

Matching the expectation, we have \( \mathbb{E}\left(\frac{a}{S}\right) = a \mathbb{E}\left(\frac{1}{S}\right) = \lambda \). Since \( S \sim \text{Gamma}(n,\lambda) \), \( \mathbb{E}(S) = \frac{n}{\lambda} \). By properties of the gamma distribution, \( \mathbb{E}\left(\frac{1}{S}\right) = \frac{\lambda}{n-1} \). Solving \( a\frac{\lambda}{n-1} = \lambda \), gives \( a = n-1 \). Thus, the unbiased estimator is \( W = \frac{n-1}{S} \).

Calculating Fisher Information

The observed Fisher Information for one sample of the exponential distribution is \( \mathcal{I}(\lambda) = \frac{1}{\lambda^{2}} \). For \( n \) independent samples, \( \mathcal{I}(\lambda) = \frac{n}{\lambda^{2}} \).

Showing Expected Value of \(W^2\)

For \( W = \frac{n-1}{S} \), we need to compute \( \mathbb{E}(W^2) = \mathbb{E}\left(\left(\frac{n-1}{S}\right)^2\right) \). The required \( \mathbb{E}\left(S^{-2}\right) = \frac{\lambda^2}{(n-2)(n-1)} \) using gamma distribution properties. Thus, \( \mathbb{E}(W^2) = \frac{(n-1)^2\lambda^2}{(n-1)(n-2)} = \frac{(n-1)\lambda^2}{n-2} \).

Cramér-Rao Lower Bound

The Cramér-Rao Lower Bound (CRLB) for an unbiased estimator of \( \lambda \) is \( \frac{1}{n\mathcal{I}(\lambda)} = \frac{\lambda^2}{n} \). We observe that \( \mathbb{E}(W^2) eq \text{CRLB} \), indicating that the estimator does not attain the CRLB. However, evaluating the asymptotic behavior as \( n \to \infty \), \( \frac{n-1}{n-2} \to 1 \), showing \( W \) is asymptotically efficient.

Unbiased Estimator for \( \psi \)

Given \( \psi = \operatorname{Pr}(Y>a) \), the indicator function \( I(Y_1 > a) \) is one if \( Y_1 > a \) and zero otherwise. It is an unbiased estimator of \( \psi \) since \( \mathbb{E}[I(Y_1 > a)] = \operatorname{Pr}(Y_1 > a) = \psi \).

Minimum Variance Unbiased Estimator (MVUE)

The MVUE of \( \psi = e^{-\lambda a} \) is the sample mean of \( I(Y_i > a) \) for \( i=1,2,\ldots,n \). However, it does not attain the Cramér-Rao lower bound, a common result for indicator functions, because MVUEs for probabilities are discrete and often cannot achieve CRLB.

Key concepts

Exponential Distribution

The exponential distribution is a continuous probability distribution commonly used to model the time between events in a Poisson process. It is characterized by a single parameter, \( \lambda \), which is the rate parameter, essentially describing the frequency at which the events occur. The probability density function (PDF) of an exponential distribution is given by:\[ f(y; \lambda) = \lambda e^{-\lambda y}, \quad y > 0 \]Here are some key points about the exponential distribution:

• It is memoryless, meaning the probability of an event occurring in the next interval is independent of any prior intervals.
• The mean or expected value of the exponential distribution is \( 1/\lambda \), and the variance is \( 1/\lambda^{2} \).
• It is a special case of the gamma distribution, with the shape parameter equal to 1.

This distribution is especially useful for modeling waiting times and life testing scenarios.

Fisher Information

The Fisher Information is a concept that describes the amount of information that an observable random variable carries about an unknown parameter upon which the likelihood function depends. For the exponential distribution, a single observed Fisher Information is given by:\[ \mathcal{I}(\lambda) = \frac{1}{\lambda^2} \]This value indicates that as the rate \( \lambda \) increases, the information we have about the parameter decreases. When considering a sample of size \( n \), the total Fisher Information becomes:\[ \mathcal{I}(\lambda) = \frac{n}{\lambda^2} \]Understanding Fisher Information helps in assessing the efficiency of an estimator, ensuring that it can potentially converge to the actual parameter as accurately as possible when more data is available.

Cramér-Rao Lower Bound

The Cramér-Rao Lower Bound (CRLB) provides a lower bound on the variance of unbiased estimators of a parameter. It is important because it sets a benchmark for how good an estimator can theoretically be. For an unbiased estimator of \( \lambda \), the CRLB is expressed as:\[ \text{CRLB} = \frac{1}{n \mathcal{I}(\lambda)} = \frac{\lambda^2}{n} \]This means that any unbiased estimator of \( \lambda \) will have a variance at least as large as this bound. In our problem, the estimator \( W = \frac{n-1}{S} \) does not attain the CRLB, though it becomes asymptotically efficient due to its variance approaching the CRLB as the sample size \( n \) increases. This behavior illustrates that while some estimators may not be efficient in finite samples, they can still perform well as the dataset grows.

Unbiased Estimator

An unbiased estimator is a statistical estimate where the expected value of the estimate equals the actual parameter value being estimated. In simple terms, an unbiased estimator, on average, hits the true parameter value. For example, in this exercise, the estimator \( W = \frac{n-1}{S} \) is unbiased for the parameter \( \lambda \) of an exponential distribution because:\[ \mathbb{E}\left(W\right) = \lambda \]An unbiased estimator is crucial because it ensures no systematic deviation from the true parameter value. Finding such estimators often involves leveraging properties of probability distributions and known parameters, so the expected value of the estimator aligns perfectly with the parameter in question.