Question

Find maximum likelihood estimates for \(\theta\) based on a random sample of size \(n\) from the densities (i) \(\theta y^{\theta-1}, 00 ;\) (ii) \(\theta^{2} y e^{-\theta y}, y>0, \theta>0 ;\) and (iii) \((\theta+1) y^{-\theta-2}\), \(y>1, \theta>0\)

Short answer

(i) \( \hat{\theta} = -\frac{n}{\sum \log y_i} \), (ii) \( \hat{\theta} = \frac{2n}{\sum y_i} \), (iii) \( \hat{\theta} = \frac{n}{\sum \log y_i} - 1 \).

Step-by-step solution

Setup for Density (i)

Given the density function for (i), \( f(y; \theta) = \theta y^{\theta-1} \). The likelihood function for a sample \( y_1, y_2, \ldots, y_n \) is the product of the densities: \( L(\theta) = \prod_{i=1}^n \theta y_i^{\theta-1} = \theta^n \prod_{i=1}^n y_i^{\theta-1} \). Simplifying gives \( L(\theta) = \theta^n \prod_{i=1}^n y_i^{\theta-1} = \theta^n \cdot (\prod_{i=1}^n y_i)^{\theta-1} \).

Log-Likelihood for Density (i)

The log-likelihood is \( \log L(\theta) = n \log \theta + (\theta - 1) \sum_{i=1}^n \log y_i \).

Differentiate and Solve for Density (i)

Differentiate the log-likelihood with respect to \( \theta \): \( \frac{d}{d\theta} \log L(\theta) = \frac{n}{\theta} + \sum_{i=1}^n \log y_i \). Set it equal to zero: \( \frac{n}{\theta} + \sum_{i=1}^n \log y_i = 0 \). Solving for \( \theta \), we find \( \hat{\theta} = -\frac{n}{\sum_{i=1}^n \log y_i} \).

Setup for Density (ii)

For (ii), the density is \( f(y; \theta) = \theta^2 y e^{-\theta y} \). For a sample \( y_1, y_2, \ldots, y_n \), the likelihood function is \( L(\theta) = \theta^{2n} \prod_{i=1}^n y_i e^{-\theta \sum_{i=1}^n y_i} \).

Log-Likelihood for Density (ii)

The log-likelihood is \( \log L(\theta) = 2n \log \theta + \sum_{i=1}^n \log y_i - \theta \sum_{i=1}^n y_i \).

Differentiate and Solve for Density (ii)

Differentiate the log-likelihood: \( \frac{d}{d\theta} \log L(\theta) = \frac{2n}{\theta} - \sum_{i=1}^n y_i \). Setting it to zero gives \( \frac{2n}{\theta} - \sum_{i=1}^n y_i = 0 \). The MLE for \( \theta \) is \( \hat{\theta} = \frac{2n}{\sum_{i=1}^n y_i} \).

Setup for Density (iii)

For (iii), the density is \( f(y; \theta) = (\theta+1) y^{-\theta-2} \), with the likelihood \( L(\theta) = (\theta+1)^n \prod_{i=1}^n y_i^{-\theta-2} \).

Log-Likelihood for Density (iii)

The log-likelihood is \( \log L(\theta) = n \log(\theta+1) - (\theta+2) \sum_{i=1}^n \log y_i \).

Differentiate and Solve for Density (iii)

Differentiate the log-likelihood: \( \frac{d}{d\theta} \log L(\theta) = \frac{n}{\theta+1} - \sum_{i=1}^n \log y_i \). Set this to zero: \( \frac{n}{\theta+1} - \sum_{i=1}^n \log y_i = 0 \). Solving for \( \theta \) gives \( \hat{\theta} = \frac{n}{\sum_{i=1}^n \log y_i} - 1 \).

Key concepts

Likelihood Function

The likelihood function is at the core of Maximum Likelihood Estimation (MLE). It describes the probability of observing the given data under different parameter values of a statistical model. In the context of our original exercise, consider density (i) where the function provided is \( f(y; \theta) = \theta y^{\theta-1} \). For a sample \( y_1, y_2, \ldots, y_n \), the likelihood function is essentially a multiplication of the probability density functions for all observed data points:

• \(L(\theta) = \prod_{i=1}^n \theta y_i^{\theta-1} \), which simplifies to \( \theta^n \prod_{i=1}^n y_i^{\theta-1} \).

This form indicates how the likelihood function aligns with the data given the parameter \( \theta \). By maximizing this function, MLE helps in estimating the most likely value of \( \theta \) that could have resulted in the observed data.

Log-Likelihood

The log-likelihood is derived from the likelihood function, representing a transformation to simplify calculations. Particularly useful for products, the logarithmic transformation turns these into sums. This process is evident in solving our original exercise.

• In density (i), we transform the likelihood \( L(\theta) \) to a log-likelihood: \( \log L(\theta) = n \log \theta + (\theta - 1) \sum_{i=1}^n \log y_i \).
• This formulation makes differentiation more manageable, especially when dealing with large sample sizes or complex likelihoods.

Log-likelihood only changes the scale, not the solution, which is why maximizing log-likelihood gives the same result as maximizing the likelihood directly.

Differentiation

With the log-likelihood function in hand, differentiation becomes a key tool to find the maximum likelihood estimates. Differentiation, in mathematical terms, involves computing the derivative of a function.

• The main goal is to find critical points by setting the derivative to zero and solving for \( \theta \).
• For density (i), differentiate the log-likelihood \( \frac{d}{d\theta} \log L(\theta) = \frac{n}{\theta} + \sum_{i=1}^n \log y_i \).

Setting this expression equal to zero allows us to solve for \( \theta \), locating the parameter value that maximizes the likelihood. This process highlights how calculus underpins statistical estimation methods.

Probability Density Function

The probability density function (PDF) provides a way to describe the distribution of continuous random variables. In MLE, the PDF helps define the likelihood function for given data.

• It specifies the relative likelihood of different outcomes of a random variable.
• For instance, in our exercise, density (i) uses a PDF \( f(y; \theta) = \theta y^{\theta-1} \), valid in the interval \( 0 < y < 1 \).

This function is fundamental in asserting properties of the data and creating models that reflect the underlying probability theories. Understanding PDFs allows us to grasp how data is expected to behave and aids in deriving related statistical properties.