Question

Given the pdf $$f(x ; \theta)=\frac{1}{\pi\left[1+(x-\theta)^{2}\right]}, \quad-\infty

Short answer

The Rao-Cramér lower bound is $2/n$. Asymptotically as $n$ goes to infinity, the distribution of $\sqrt{n}(\hat{\theta }-\theta )$ becomes standard normal $N(0,1)$.

Step-by-step solution

Calculation of Fisher Information

From the given p.d.f., we first compute the first derivative of the log-likelihood $\log f(x;\theta )$ w.r.t. $\theta$. Then calculate the Fisher Information by taking expected value of its square.

Find the Rao-Cramér Lower Bound

Use the formula for Rao-Cramér Lower Bound which is the reciprocal of the Fisher Information. We compute this by substituting the calculated Fisher Information into this formula.

Find the MLE

To compute the MLE, we first write down the likelihood function, which is the product of the density functions. Then take a log and differentiate w.r.t. $\theta$. Equate this to zero and solve for $\theta$. The result is $\hat{\theta }$.

Determine the Asymptotic Distribution

We can use standard results regarding the asymptotic properties of the MLE, which state that as $n$ approaches infinity, $\sqrt{n}(\hat{\theta }-\theta )$ converges in distribution to a normal distribution with mean 0 and variance equal to the reciprocal of the Fisher Information.

Key concepts

Rao-Cramér Lower Bound

The Rao-Cramér lower bound provides a way to set a lower limit on the variance of an unbiased estimator. It's a crucial concept in statistics, especially when dealing with parameter estimation. For the Cauchy distribution in our exercise, the goal is to show that this bound is $\frac{2}{n}$, where $n$ is the sample size.
To determine this, you utilize the Fisher Information (FI). The bound is essentially the inverse of the FI, representing the best accuracy you can hope to achieve when estimating a parameter like $\theta$.

• Start with calculating the Fisher Information from the log-likelihood of the distribution.
• The Rao-Cramér lower bound formula is $\frac{1}{I(\theta )}$, where $I(\theta )$ is the Fisher Information.
• By computing, you find that this lower bound indeed equals $\frac{2}{n}$.

This tells us how precise any unbiased estimator could get, essentially setting a benchmark for estimation.
Understanding these limits is highly useful in comparing different estimation methods.

Fisher Information

Fisher Information measures the amount of information that an observable variable carries about an unknown parameter. It's central in statistical parameter estimation.
For the Cauchy distribution, Fisher Information helps us find the accuracy limit of our estimator for $\theta$. Here's how it typically works:

• Calculate the likelihood function: The density function of the Cauchy distribution is simplified to a log-likelihood to ease differentiation.
• Find the derivative: Differentiate the log-likelihood with respect to $\theta$.
• Take the square and expectation: Squaring the derivative and finding its expectation gives us the Fisher Information $I(\theta )$.

In this case, Fisher Information helps us confirm that the variance of any unbiased estimator for $\theta$ has a minimum bound. This minimizes the error and precision we can aspire to achieve, represented as $\frac{2}{n}$ in our exercise.
It’s fundamentally like knowing how sharp your tools are before trying to carve out a solution.

Maximum Likelihood Estimation

Maximum Likelihood Estimation (MLE) is a method used to estimate the parameters of a statistical model. It finds the parameter values that make the observed data most probable.
For the Cauchy distribution, here's how MLE works:

• Formulate the likelihood function: Multiply the density functions for the observed data.
• Optimize using logs: Convert to log-likelihood to simplify the math.
• Derive and solve: Differentiate with respect to $\theta$ and solve for $\theta$ by setting the derivative to zero.

The solution, $\hat{\theta }$, is our MLE for $\theta$, representing the most likely value given the data.
But there's more: As the sample size $n$ increases, $\sqrt{n}(\hat{\theta }-\theta )$ will approach a normal distribution. It has a mean of zero and variance equal to the inverse Fisher Information.
This indicates that with more data, the estimates become more precise and reliable, which is a reassuring aspect of using MLE, especially in practical applications.