Question

Let \(X_{1}, \ldots, X_{n}\) be iid with pdf \(f(x ; \theta)=1 /(3 \theta),-\theta0\). (a) Find the mle \(\hat{\theta}\) of \(\theta\). (b) Is \(\widehat{\theta}\) a sufficient statistic for \(\theta\) ? Why? (c) Is \((n+1) \widehat{\theta} / n\) the unique MVUE of \(\theta\) ? Why?

Short answer

\(\hat{\theta} = \max\{X_1, ..., X_n\}\) is the MLE and is also a sufficient statistic for \(\theta\). \(\widehat{\theta}^{*} = \frac{(n+1)\widehat{\theta}}{n}\) is the unique MVUE of \(\theta\).

Step-by-step solution

Compute the likelihood function

Start by writing the likelihood function. Given the probability density function (PDF) \(f(x ; \theta)=1 /(3 \theta),-θ<x<2 θ\), the likelihood function is \(L(\theta ; x)=\prod_{i=1}^{n} f\left(x_{i} ; \theta\right) = \prod_{i=1}^{n} 1 / (3 \theta) = \frac{1}{(3\theta)^n}\) for \(-\theta < x < 2\theta\).

Compute the MLE

To find the maximum likelihood estimator (MLE) \(\hat{\theta}\), we need to maximize the likelihood function. However, because it's easier to work with sums than products, maximize the log-likelihood instead. The log-likelihood function is \(\ln(L(\theta ; x)) = \sum_{i=1}^{n} \ln(f(x_{i} ; \theta)) = -n \ln(3\theta)\). Now find the derivative of \(\ln(L(\theta ; x))\) with respect to \(\theta\), set it equal to zero and solve for \(\theta\). Because the range of \(x\) depends on \(\theta\), we find that \(\hat{\theta} = \max\{X_1, ..., X_n\}\), where \(X_i\) are random variables.

Check if the MLE is a sufficient statistic

We use the Neyman Factorization Theorem to test if a statistic is sufficient. A statistic \(T(X)\) is sufficient for \(\theta\) if the likelihood function can be factored into a product of two functions, one depending only on \(T(X)\) and \(\theta\), and another only on \(X\). Here, the likelihood function is \(\frac{1}{(3\theta)^n}\) for \(-\theta < x < 2\theta = g(T, \theta)h(x)\), where \(T = \max\{X_1, ..., X_n\}\) and \(g(T, \theta) = \frac{1}{(3\theta)^n}\) for \(-\theta < T < 2\theta\) and \(h(x)=1\) which verifies that \(\hat{\theta}\) is a sufficient statistic.

Check if the adjusted estimator is the MVUE

We can check if \(\widehat{\theta}^{*} = \frac{(n+1)\widehat{\theta}}{n}\) is MVUE by verifying it's unbiased and of minimum variance. Test for unbiasedness: \(E[\widehat{\theta}^{*}] = E\left[\frac{(n+1)}{n}\max\{X_{1}, \ldots, X_{n}\}\right] = \theta\). So \(\widehat{\theta}^{*}\) is unbiased. Since \(\widehat{\theta}^{*}\) is a function of \(\widehat{\theta}\), which is a sufficient statistic, by the Rao-Blackwell Theorem, \(\widehat{\theta}^{*}\) must also be a UMVUE. Therefore, \(\widehat{\theta}^{*}\) is the unique MVUE of \(\theta\).

Key concepts

Likelihood Function

The likelihood function is a fundamental concept in statistics, providing a link between a statistical model and observed data. It's a function of the parameters of a model, given specific observed data. For a set of independent and identically distributed (iid) random variables with a probability density function (PDF), the likelihood function is the product of the PDFs evaluated at the observed data points.

Considering the exercise provided, where the PDF is given by f(x ; \theta)=1 /(3\theta) for -\theta, the likelihood function is calculated by taking the product of this PDF over all observations, \(L(\theta ; x) = \frac{1}{(3\theta)^n}\) in the specified range. To find the maximum likelihood estimator (MLE), we must find the parameter value that maximizes this likelihood function. Typically, we use the natural logarithm to transform the product into a sum, easing differentiation and solving for the estimator.

Sufficient Statistic

A sufficient statistic is a summary of the data that contains all the information needed to compute the likelihood function for parameter estimation. The significance of a sufficient statistic is its ability to compress data without losing information regarding the parameter of interest.

Utilizing the Neyman Factorization Theorem, we can determine whether a statistic is sufficient. In the exercise, the maximum likelihood estimator \(\hat{\theta}\) derived is a function of the maximum observed value in the sample and can be used to factor the likelihood function. Thus, it serves as a sufficient statistic for \(\theta\) because it fully encapsulates the information from the sample necessary for parameter estimation. The function can be expressed as a product of a function g(T, \theta) that depends on \(\hat{\theta}\) and \(\theta\), and a function h(x) that depends only on the data, demonstrating the sufficiency of \(\hat{\theta}\).

Minimum Variance Unbiased Estimator (MVUE)

The minimum variance unbiased estimator (MVUE) refers to an estimator that has the lowest variance among all unbiased estimators of a parameter. Unbiasedness indicates that the expected value of the estimator equals the true parameter value, while minimum variance signifies that it is the most precise estimator possible.

In the context of the exercise, we analyze \(\widehat{\theta}^{*} = \frac{(n+1)\widehat{\theta}}{n}\) as a potential MVUE for \(\theta\). To qualify as MVUE, this estimator must be both unbiased and exhibit the minimum possible variance. Due to \(\widehat{\theta}\) being a sufficient statistic, and leveraging the Rao-Blackwell Theorem, we demonstrate that the adjusted estimator \(\widehat{\theta}^{*}\) is indeed an MVUE. It's obtained by improving on the MLE using the information from the sufficiency, thus ensuring it not only provides an unbiased prediction of \(\theta\) but does so with the smallest variance attainable, confirming its uniqueness as MVUE within the bounds of this statistical model.