Question

A random variable \(X\) has PDF $$ f(x ; \theta)=\frac{1}{2} e^{-|x-\theta|}, \quad-\infty

Short answer

The Maximum Likelihood estimate for parameter \( \theta \) is the sample median, which is the middle value of the sorted sample \(X_{1}, X_{2}, ... , X_{n}\). If the sample size \(n\) is even, \( \theta \) is the average of the two middle observations.

Step-by-step solution

Write down the likelihood function

The likelihood function for a random sample \( X_{1}, X_{2}, \ldots, X_{n} \) from a population is the joint probability density (or mass) function of the observations. For independent observations, this is the product of the individual density (or mass) functions. So in this case, using the given probability density function \( f(x ; \theta)=\frac{1}{2} e^{-|x-\theta|} \), the likelihood function will be: \(L(\theta) = \prod_{i=1}^{n}f(x_i; \theta) = \prod_{i=1}^{n}\frac{1}{2} e^{-|x-\theta|}\).

Logarithm of the Likelihood function

To simplify the optimization problem, its common practice to take the natural logarithm of the likelihood function, known as the log-likelihood function. The logarithm is a strictly increasing function, so any value which maximizes the log-likelihood function, also maximizes the likelihood function. The log-likelihood function is given by: \(l(\theta) = \log(L(\theta)) = \sum_{i=1}^{n}\log(f(x_i; \theta)) = \sum_{i=1}^{n}\log(\frac{1}{2} e^{-|x-\theta|}) = -n\log(2) - \sum_{i=1}^{n}|x-\theta|\).

Extremum of the log-likelihood function

Now the goal is to maximize \(l(\theta)\). But maximization of \(l(\theta)\) is equivalent to minimization of \(\sum_{i=1}^{n}|x-\theta|\). However, it is known that the sum of absolute deviations reaches its minimum when the deviating point is a median of the sample values. Thus, to solve this problem one must compute the sample median.

Compute the Sample Median

Sort the sample observations \(x_1, x_2, ..., x_n\) in ascending order and identify the middle value. If \(n\) is odd, the sample median \(M\) is the middle observation. If \(n\) is even, the sample median \(M\) is the average of the two middle observations.

Key concepts

Probability Density Function

Before diving into maximum likelihood estimation, it's essential to understand the concept of a probability density function (PDF). PDFs are used to describe the likelihood of a continuous random variable taking a particular value. Unlike probabilities for discrete random variables, which sum up to 1, the total area under a PDF curve is 1.
For our problem, the provided PDF is \( f(x ; \theta)=\frac{1}{2} e^{-|x-\theta|} \). This function is dependent on the parameter \( \theta \), which influences the shape of the distribution.

Some key points about PDFs include:

• The function must be non-negative for all values of its variable.
• The integral over all possible values of the variable must equal 1.
• The PDF does not, by itself, provide the probability of the variable taking any particular value, as it's used for continuous variables.

Log-Likelihood Function

In our exercise, we're tasked with estimating the parameter \( \theta \) using maximum likelihood estimation (MLE). One of the crucial steps in MLE is forming the likelihood function, which in this case, involves the joint probability density function for all sample observations.

To derive the MLE, it's convenient to transform the likelihood function using its natural logarithm, resulting in the log-likelihood function. The transformation is common because it:

• Converts products into sums, simplifying the differentiation process.
• Retains the location of the maximum, as the logarithmic function is monotonically increasing.

For our scenario, the log-likelihood function is given by:
\[ l(\theta) = -n\log(2) - \sum_{i=1}^{n}|x_i-\theta| \]
This expression highlights the dependency of the likelihood on \( \theta \), making it possible to estimate \( \theta \) by optimizing the log-likelihood.

Sample Median

After expressing the log-likelihood function, the task shifts to finding the parameter \( \theta \) that maximizes this function. For our specific form of the log-likelihood, maximizing it is equivalent to minimizing the sum \( \sum_{i=1}^{n}|x_i-\theta| \).

This is a classic optimization problem, where the solution is the sample median. The sample median minimizes the sum of absolute deviations from the sample observations, which is why it plays a vital role in this context.

To calculate the sample median:

• Sort the sample observations \( x_1, x_2, \ldots, x_n \) in ascending order.
• If \( n \) is odd, the median is the middle observation.
• If \( n \) is even, the median is the average of the two middle observations.

By identifying the sample median, we effectively determine the maximum likelihood estimate for \( \theta \). This step is crucial as it converts the problem into one that leverages the statistical properties of the median.