Question

(a) Let \(y_{1}, \ldots, y_{n}\) be a Poisson random sample with mean \(\theta\), and suppose that the prior density for \(\theta\) is gamma, $$ \pi(\theta)=g(\theta ; \alpha, \lambda)=\frac{\lambda^{\alpha} \theta^{\alpha-1}}{\Gamma(\alpha)} \exp (-\lambda \theta), \quad \theta>0, \lambda, \alpha>0 $$ Show that the posterior density of \(\theta\) is \(g\left(\theta ; \alpha+\sum y_{j}, \lambda+n\right)\), and find conditions under which the posterior density remains proper as \(\alpha \downarrow 0\) even though the prior density becomes improper in the limit. (b) Show that \(\int \theta g(\theta ; \alpha, \lambda) d \theta=\alpha / \lambda\). Find the prior and posterior means \(\mathrm{E}(\theta)\) and \(\mathrm{E}(\theta\) ) \(y\) ), and hence give an interpretation of the prior parameters. (c) Let \(Z\) be a new Poisson variable independent of \(Y_{1}, \ldots, Y_{n}\), also with mean \(\theta .\) Find its posterior predictive density. To what density does this converge as \(n \rightarrow \infty\) ? Does this make sense?

Short answer

Posterior density is gamma with parameters \( \alpha + \sum y_i, \lambda + n \). Prior mean is \( \frac{\alpha}{\lambda} \), posterior mean is \( \frac{\alpha + \sum y_i}{\lambda + n} \). Posterior predictive density is negative binomial.

Step-by-step solution

Define Likelihood Function

The likelihood function for a Poisson random sample with mean \( \theta \) is given by \( L(\theta) = \prod_{i=1}^{n} \frac{\theta^{y_i} e^{-\theta}}{y_i!} = \theta^{\sum y_i} e^{-n\theta} \cdot \text{(constant)} \). Simplifying this, it becomes \( \theta^{\sum y_i} e^{-n\theta} \).

Define Prior Density

The prior density for \( \theta \) is given by a gamma distribution: \( \pi(\theta) = \frac{\lambda^{\alpha} \theta^{\alpha-1}}{\Gamma(\alpha)} e^{-\lambda \theta} \).

Compute Posterior Density

Combine the likelihood and the prior to compute the posterior density. The posterior density is proportional to the product: \( \theta^{\sum y_i} e^{-n\theta} \cdot \theta^{\alpha - 1} e^{-\lambda \theta} = \theta^{\alpha + \sum y_i - 1} e^{-(\lambda + n)\theta} \). This is the kernel of a gamma distribution with parameters \( \alpha + \sum y_i \) and \( \lambda + n \).

Check Posterior Properness

For the posterior density to remain proper despite an improper prior as \( \alpha \downarrow 0 \), the sum \( \sum y_i \) must be positive, as this requirement ensures that the shape parameter of the gamma distribution remains positive.

Calculate Expectation of Gamma Distribution

The expectation of a gamma distribution \( g(\theta; \alpha, \lambda) \) is given by \( \mathrm{E}(\theta) = \frac{\alpha}{\lambda} \).

Find Prior and Posterior Means

The prior mean is \( \mathrm{E}(\theta) = \frac{\alpha}{\lambda} \). The posterior mean, given the posterior parameters \( \alpha + \sum y_i \) and \( \lambda + n \), is \( \mathrm{E}(\theta \mid y) = \frac{\alpha + \sum y_i}{\lambda + n} \). The parameter \( \alpha \) indicates prior information on the mean, and \( \lambda \) reflects prior belief in precision.

Determine Posterior Predictive Density

For a new observation \( Z \) with a Poisson mean \( \theta \), the posterior predictive density \( P(Z = z \mid y) \) follows a negative binomial distribution as \( \text{NB}(z; \alpha + \sum y_i, \frac{n}{\lambda + n}) \). As \( n \to \infty \), it converges to a Poisson distribution with mean equal to the sample mean, reflecting the law of large numbers.

Interpret the Result

The predictive density convergence makes sense as, with increasing sample size, the data provides most of the information about \( \theta \), decreasing the influence of the prior.

Key concepts

Poisson Distribution

The Poisson distribution is a widely used probability distribution for modeling the number of events occurring within a fixed period of time or space. It's characterized by its parameter \( \theta \), which is the mean or expected number of events. A Poisson random variable \( Y \) represents the count of these events, and its probability mass function is given by:

• \( P(Y = y) = \frac{\theta^y e^{-\theta}}{y!} \) for \( y = 0, 1, 2, \ldots \)

This distribution is particularly useful when dealing with rare events where the mean and variance are equal. In the context of Bayesian inference, the Poisson distribution serves as the likelihood function when modeling data believed to follow this distribution type.

Gamma Distribution

The gamma distribution is a continuous probability distribution often used as a prior in Bayesian statistics, particularly for parameters that represent rates or scales, such as the mean rate in a Poisson process. Characterized by two parameters, \( \alpha \) (shape) and \( \lambda \) (rate), its probability density function is:

• \( g(\theta ; \alpha, \lambda) = \frac{\lambda^{\alpha} \theta^{\alpha-1}}{\Gamma(\alpha)} \exp(-\lambda \theta) \), where \( \theta > 0 \)

The parameter \( \alpha \) relates to the weight of prior information, while \( \lambda \) represents the prior expected precision. In Bayesian analysis, choosing a gamma distribution as a prior for a Poisson mean (like \( \theta \) ) appropriately reflects prior beliefs about the rate of events. It is flexible and computationally convenient, as it combines with a Poisson likelihood to yield another gamma distribution.

Posterior Distribution

The posterior distribution in Bayesian inference combines prior beliefs with new evidence from data. For our Poisson mean \( \theta \), given a gamma prior, the posterior distribution is also gamma due to the conjugate prior relationship. This combined expression is:

• \( g(\theta ; \alpha + \sum y_i, \lambda + n) \)

Here, \( \alpha + \sum y_i \) and \( \lambda + n \) are the posterior shape and rate parameters, respectively. This update is based on new data summary presented by \( \sum y_i \) and the number of data points \( n \). The posterior allows us to refine our belief about \( \theta \), balancing prior information with data-derived insights. It's critical for making informed decisions using both prior knowledge and new observations.

Predictive Density

The predictive density provides a way to forecast future observations based on the current model and data. For a new Poisson variable \( Z \) with mean \( \theta \), the predictive density incorporates both the observed data and the prior beliefs, providing a probabilistic model for new data points:

• \( P(Z = z \mid y) \approx \text{NB}(z; \alpha + \sum y_i, \frac{n}{\lambda + n}) \)

Where NB stands for the Negative Binomial distribution, reflecting the uncertainty in \( \theta \). With increasing sample size \( n \), the influence of the prior diminishes, and this density converges to a Poisson distribution centered on observed data. This convergence illustrates how more data leads to predictions grounded in empirical evidence, aligning with the concept of the law of large numbers.