Question

Consider the Bayes model \(X_{i} \mid \theta, i=1,2, \ldots n \sim\) iid with distribution Poisson \((\theta), \theta>0\) $$ \Theta \sim h(\theta) \propto \theta^{-1 / 2} $$ (a) Show that \(h(\theta)\) is in the class of Jeffrys priors. (b) Show that the posterior pdf of \(2 n \theta\) is the pdf of a \(\chi^{2}(2 y+1)\) distribution, where \(y=\sum_{i=1}^{n} x_{i}\) (c) Use the posterior pdf of Part (b) to obtain a \((1-\alpha) 100 \%\) credible interval for \(\theta\) (d) Use the posterior pdf in Part (d) to determine a Bayesian test for the hypotheses \(H_{0}: \theta \geq \theta_{0}\) versus \(H_{1}: \theta<\theta_{0}\), where \(\theta_{0}\) is specified.

Short answer

The given prior belongs to the class of Jeffrey's priors as it is a function of the square root of the Fisher Information. The posterior pdf for \(2n\theta\) follows a chi-square distribution with \(2y+1\) degrees of freedom. The credible interval for \(\theta\) was obtained as \([\frac{\chi^{2}_{\alpha/2, 2y+1}}{2n}, \frac{\chi^{2}_{1-\alpha/2, 2y+1}}{2n}]\). The Bayesian Test involves comparing \(\theta' = \frac{\chi^{2}_{\alpha, 2y+1}}{2n}\) to \(\theta_{0}\) to decide whether to reject the null hypothesis.

Step-by-step solution

Deriving Jeffrey's priors

The Jeffrey's prior for a parameter \(\theta\) can be defined as \(h(\theta) = c \times I^{1/2}( \theta)\), where \(I( \theta)\) is the Fisher Information for \(\theta\) and \(c\) is the normalization constant. For a Poisson distribution \(X_{i} \mid \theta, i=1,2, \ldots n\), the Fisher Information is \(I(\theta) = 1/\theta\). The square root of this yields \(I^{1/2}(\theta) = \theta^{-1/2}\). Therefore, \(h(\theta)\) is indeed in the class of Jeffrey's priors.

Deriving the posterior probability density function

The posterior distribution of a Bayes model is proportional to the product of the prior and likelihood functions. This leads to the following expression for the posterior pdf of \(2n\theta\): \(h(2n\theta) \propto (2n\theta)^{-1/2}\) X \(e^{-2n\theta}\). Besides, it is known that the pdf of a chi-square distribution with \(u\) degrees of freedom is: \(f(x) \propto x^{(u-2)/2} e^{-x/2}\) for \(x>0\). By comparing this expression with the derived posterior pdf, it can be concluded that this pdf follows a chi-square distribution with \(2y+1\) degrees of freedom, where \(y = \sum_{i=1}^{n} x_{i}\).

Obtaining a credible interval

A \(100(1-\alpha)%\) credible interval for a chi-square distribution with \(u\) degrees of freedom is given by the interval between the \(\alpha/2\) and \(1-\alpha/2\) quantiles of that distribution, denoted \(\chi^{2}_{\alpha/2, u}\) and \(\chi^{2}_{1-\alpha/2, u}\) respectively. Therefore, a \(100(1-\alpha)%\) credible interval for \(2n\theta\) is \([\chi^{2}_{\alpha/2, 2y+1},\chi^{2}_{1-\alpha/2, 2y+1}]\). Since \(\theta = \frac{y}{n}\), this interval can be rewritten as \([\frac{\chi^{2}_{\alpha/2, 2y+1}}{2n}, \frac{\chi^{2}_{1-\alpha/2, 2y+1}}{2n}]\).

Determining a Bayesian test

The given null hypothesis is \(H_{0}: \theta \geq \theta_{0}\) and the alternative hypothesis is \(H_{1}: \theta<\theta_{0}\). This translates into considering \(2n\theta' < c\) as the rejection region for the null hypothesis, where \(c\) is a constant and \(\theta' = \frac{\chi^{2}_{\alpha, 2y+1}}{2n}\). If \(\theta' < \theta_{0}\), reject the null hypothesis. If \(\theta' >= \theta_{0}\), fail to reject the null hypothesis.

Key concepts

Jeffreys Prior

In Bayesian inference, choosing a prior distribution plays a crucial role. Jeffreys Prior is a popular choice due to its invariance properties and ability to handle parameter estimation objectively. It is calculated from the Fisher Information, which measures the amount of information that an observable variable carries about an unknown parameter. In the context of the Poisson distribution, Jeffreys Prior for the rate parameter \(\theta\) is determined as \( h(\theta) \propto \theta^{-1/2} \). This ensures that the prior is 'uninformative', providing no initial directional influence on the posterior distribution and thus allowing the data to speak for itself.

Posterior Distribution

The posterior distribution combines prior beliefs with evidence from the data, reflecting the updated beliefs or knowledge about the parameter after observing the data. For a Bayesian model involving Poisson-distributed data, you start with a Jeffreys Prior, given by \( h(\theta) \propto \theta^{-1/2} \), and the likelihood function. The result is a posterior that reflects a continuous PDF, often resembling a well-known distribution for practical inference. In our scenario, the posterior PDF for \( 2n \theta \) results in a \( \chi^2(2y+1) \) distribution. This transformation follows from identifying the mathematical form of the likelihood combined with the Jeffreys Prior.

Credible Interval

A credible interval in Bayesian statistics represents a range within which an unknown parameter, such as \(\theta\), lies with a certain probability, denoted as \( 1-\alpha \). Unlike frequentist confidence intervals, credible intervals naturally incorporate prior information. For our \(\chi^2\) posterior, a \(100(1-\alpha)\%\) credible interval for \(2n\theta\) can be determined between the \(\chi^2_{\alpha/2, 2y+1}\) and \(\chi^2_{1-\alpha/2, 2y+1}\) quantiles. Dividing these values by \(2n\) converts the interval back into terms of \(\theta\). This interval gives a meaningful range, considering prior and likelihood, illustrating where \(\theta\) is likely to be based on the observed data.

Bayesian Hypothesis Testing

In Bayesian analysis, hypothesis testing is used to compare different hypotheses based on observed data. For our exercise, we are tasked to test hypotheses about \(\theta\), specifically \(H_0: \theta \geq \theta_0\) against \(H_1: \theta < \theta_0\). To achieve this, we determine a critical region by examining \(2n\theta'\) and comparing it against a critical value \(c\), which is derived from the \(\chi^2\) posterior distribution for a given significance level \(\alpha\). If \(\theta' < \theta_0\), evidence suggests rejecting \(H_0\), indicating that the data supports the alternative hypothesis.

Poisson Distribution

The Poisson distribution is an essential statistical distribution used to model count data and rare events over a fixed interval. When involving Bayesian statistics, it plays a critical role in forming the likelihood component of the model. The parameter \(\theta\) represents the average rate at which events occur. The Poisson distribution, characterized by its discrete nature and dependence on \(\theta\), is pivotal for understanding event probabilities over time or space. Specifically, in Bayesian contexts, modeling with the Poisson distribution assists in examining event frequencies, ultimately impacting the formation of prior and posterior distributions.

Chi-Square Distribution

The chi-square distribution is a continuous distribution important in statistics, especially in hypothesis testing and constructing confidence (or credible) intervals. In our Bayesian framework, the posterior PDF for \(2n\theta\) emerged as a chi-square distribution with \(2y+1\) degrees of freedom. This property allows us to derive both credible intervals and decision rules for hypothesis testing with ease. The chi-square distribution is shaped by the degrees of freedom, which determine the variance and skewness. Knowing these properties makes it useful for a wide range of statistical applications, from goodness-of-fit tests to Bayesian analysis, where dealing with squared normal (or chi-squared) distributions is commonplace.