Question

Let \(X_{1}, X_{2}, \ldots, X_{n}\) be a random sample from a Poisson distribution with mean \(\theta>0\) (a) Statistician \(A\) observes the sample to be the values \(x_{1}, x_{2}, \ldots, x_{n}\) with sum \(y=\sum x_{i} .\) Find the mle of \(\theta\). (b) Statistician \(B\) loses the sample values \(x_{1}, x_{2}, \ldots, x_{n}\) but remembers the sum \(y_{1}\) and the fact that the sample arose from a Poisson distribution. Thus \(B\) decides to create some fake observations which he calls \(z_{1}, z_{2}, \ldots, z_{n}\) (as he knows they will probably not equal the original \(x\) -values) as follows. He notes that the conditional probability of independent Poisson random variables \(Z_{1}, Z_{2}, \ldots, Z_{n}\) being equal to \(z_{1}, z_{2}, \ldots, z_{n}\), given \(\sum z_{i}=y_{1}\) is $$\frac{\frac{\theta^{x_{1}} e^{-\theta}}{z_{1} !} \frac{\theta^{x_{2}} e^{-\theta}}{z_{2} !} \cdots \frac{\theta^{x_{n}} e^{-\theta}}{x_{n} !}}{\frac{(n \theta)^{y_{1}} e^{n \theta}}{y_{1} !}}=\frac{y_{1} !}{z_{1} ! z_{2} ! \cdots z_{n} !}\left(\frac{1}{n}\right)^{z_{1}}\left(\frac{1}{n}\right)^{z_{2}} \cdots\left(\frac{1}{n}\right)^{z_{m}}$$ since \(Y_{1}=\sum Z_{i}\) has a Poisson distribution with mean \(n \theta .\) The latter distribution is multinomial with \(y_{1}\) independent trials, each terminating in one of \(n\) mutually exclusive and exhaustive ways, each of which has the same probability \(1 / n .\) Accordingly, \(B\) runs such a multinomial experiment \(y_{1}\) independent trials and obtains \(z_{1}, z_{2}, \ldots, z_{n} .\) Find the likelihood function using these \(z\) values. Is it proportional to that of statistician \(A ?\) Hint: Here the likelihood function is the product of this conditional pdf and the pdf of \(Y_{1}=\sum Z_{i}\)

Short answer

The MLE of \(\theta\) provided by statistician A is \(y/n\). Despite losing the original samples, statistician B can still estimate the MLE, because the likelihood functions of A's and B's data are proportional - they differ only by a constant not related to the parameters of the Poisson distribution.

Step-by-step solution

Calculate MLE of theta

Using the likelihood function of a Poisson distribution \(L(\theta) = \frac{(e^{-n\theta} * (\theta^{y})}{y!}\), we can find the MLE of \(\theta\) through differentiation. First, we have to take the logarithm of the likelihood function, to get \(\log(L(\theta)) = -n\theta + y\log(\theta) - \log(y!)\). Derive \(\log(L(\theta))\) and set it equal to zero, which gives us \(-n + \frac{y}{\theta} = 0\). Solving for \(\theta\), we find that the MLE of \(\theta\) is \(\hat{\theta} = y / n\).

Simulate the fake data of statistician B

Statistician B creates a fake dataset using multinomial distribution with equal probabilities for each of the n occurrences. Hence, the likelihood function for these z-values will be \(\frac{(y_1)!}{z_{1} ! z_{2} ! \cdots z_{n}!}*(1/n)^{z_{1}}*(1/n)^{z_{2}} \cdots(1/n)^{z_{n}}\). Because B knows that these z values are fake and they have been created through multinomial distribution, the likelihood function incorporates both this fact and the fact that \(\sum{z_i} = y_1\).

Compare likelihood functions of statistician A and B

Now we want to see if these likelihood functions are proportional. The only difference between the two functions is that in B's case, there is an additional factor of \(\frac{(y_1)!}{z_{1} ! z_{2} ! \cdots z_{n}!}\). However, this is a constant with respect to the parameters. Therefore, we can indeed say that the likelihood functions are proportional, since they differ only by a simple constant and the important interaction between the parameters of interest are the same, ie. the Poisson distribution shape remains the same. This allows B to make an approximate MLE of \(\theta\) despite having lost the original sample.

Key concepts

Poisson Distribution

The Poisson distribution is a probability model that describes events occurring at a constant rate within a fixed period of time or space, assuming that these events happen independently of each other. A prime characteristic of the Poisson distribution is its single parameter \(\theta\), which represents the average number of occurrences in the given interval.

In statistical terms, the Poisson distribution is deployed when we're interested in the number of times an event happens over a specified interval. For example, the number of emails a person receives in a day or the number of decay events from a radioactive source over a period of time could be modeled using a Poisson distribution if the individual events are independent and happen at a constant average rate.

The probability of observing \(x\) events in a Poisson distribution is given by the formula: \[P(X=x) = \frac{e^{-\theta} \theta^x}{x!}\] where \(e\) is the base of the natural logarithm, \(x!\) is the factorial of \(x\), and \(\theta\) is the average number of events.

Understanding the Poisson distribution is crucial when dealing with real-world scenarios that involve counting the number of events or occurrences. It helps prepare students to tackle statistical problems from a variety of disciplines, ranging from business analytics to natural sciences.

Likelihood Function

The likelihood function is a fundamental concept in statistical inference, particularly in the context of estimating parameters of a statistical model. It indicates how likely it is that the observed data occurred given specific values of the parameters. The key purpose of the likelihood function, denoted usually as \(L(\theta)\), is to provide a measure for comparing different parameter values in terms of their 'agreement' with the observed data.

For discrete distributions, such as the Poisson distribution, the likelihood function is the product of probabilities assigned to observed outcomes. Each probability is calculated using the probability mass function (PMF) of the distribution with a given parameter value.

In the exercise provided, Statistician \(A\) uses the likelihood function based on the actual data observations, while Statistician \(B\), lacking specific data points, relies on simulated data. The use of the likelihood function for parameter estimation is not limited to accurate data sets, but can also be applied to data that have been derived using assumptions, such as the fake observations by Statistician \(B\). This reflects the flexibility and utility of the likelihood function in statistical estimation.

Statistical Inference

Statistical inference encompasses the techniques that allow us to draw conclusions about a population based on a sample from that population. It often involves using sample data to estimate population parameters—such as the mean (\(\theta\)) of a Poisson distribution. Maximum likelihood estimation (MLE) is a popular method of statistical inference utilized to find the parameter value that makes the observed data most probable.

MLE assesses different candidate values for model parameters and selects the value that maximizes the likelihood function. As seen in the exercise, the MLE \(\hat{\theta} = y / n\) is determined by calculating the point where the derivative of the log-likelihood function equals zero.

Statistical inference bridges the gap between sample data and population analysis, using methods like MLE to provide reliable conclusions. It's essential for making informed decisions in uncertain conditions, making it a cornerstone of data-driven disciplines.