Question

Let \(f(x \mid \theta), \theta \in \Omega\), be a pdf with Fisher information, \((6.2 .4), I(\theta)\). Consider the Bayes model $$ \begin{aligned} X \mid \theta & \sim f(x \mid \theta), \quad \theta \in \Omega \\ \Theta & \sim h(\theta) \propto \sqrt{I(\theta)} \end{aligned} $$ (a) Suppose we are interested in a parameter \(\tau=u(\theta)\). Use the chain rule to prove that $$ \sqrt{I(\tau)}=\sqrt{I(\theta)}\left|\frac{\partial \theta}{\partial \tau}\right| . $$ (b) Show that for the Bayes model (11.2.2), the prior pdf for \(\tau\) is proportional to \(\sqrt{I(\tau)}\) The class of priors given by expression (11.2.2) is often called the class of Jeffreys' priors; see Jeffreys (1961). This exercise shows that Jeffreys' priors exhibit an invariance in that the prior of a parameter \(\tau\), which is a function of \(\theta\), is also proportional to the square root of the information for \(\tau\).

Short answer

The relationship between the Fisher information for \( \tau = u(\theta) \) and \( \theta \) is given by \( \sqrt{I(\tau)} = \sqrt{I(\theta)} \left |\frac{\partial \theta}{\partial \tau} \right | \). For the Bayes model, the prior pdf for \( \tau \) is proportional to \( \sqrt{I(\tau)} \), proving that Jeffreys' priors preserve their form under reparametrizations.

Step-by-step solution

Apply chain rule

By chain rule of differentiation, the derivative of \( \tau = u(\theta) \) with respect to \( \theta \) is given by \( \frac{\partial u}{\partial \theta} \). Now derivative of \( \theta \) with respect to \( \tau \) is the reciprocal of this, denoted by \( \frac{\partial \theta}{\partial \tau} \). Hence the reciprocal of the change in \( \theta \) for a unit change in \( \tau \) is given by \( \left | \frac{\partial \theta}{\partial \tau} \right | \).

Apply results from Step 1 to derive Fisher's information for \( \tau \)

We know that Fisher's information is invariant under reparametrizations. Therefore, if \( \tau \) is a one-to-one function of \( \theta \), then the Fisher information for \( \tau \), denoted by \( I(\tau) \), is equal to the Fisher information for \( \theta \), denoted by \( I(\theta) \), times \( \left|\frac{\partial \theta}{\partial \tau}\right|^2 \). Take square root on both sides, we get the equation required to be proved in question (a): \( \sqrt{I(\tau)} = \sqrt{I(\theta)} \left |\frac{\partial \theta}{\partial \tau} \right | \).

Prove the prior pdf for \( \tau \) is proportional to \( \sqrt{I(\tau)} \)

We are given that the prior of \( \theta \) is proportional to \( \sqrt{I(\theta)} \). Therefore, the prior for \( \tau \), a function of \( \theta \), should also be proportional to the square root of the Fisher information for \( \tau \), i.e., \( h(\tau) \propto \sqrt{I(\tau)} \). This shows that Jeffreys' priors preserve their form under reparametrizations, as is required to be shown in question (b).