Question

In Example 12.5.6, we used a hierarchical model. In that model, the parameters\({\mu _i},...,{\mu _P}\,\)were independent random variables with\({\mu _i}\)having the normal distribution with mean ψ and precision\({\lambda _0}{T_i}\,\)conditional on ψ and\({T_1},\,....{T_P}\). To make the model more general, we could also replace\({\lambda _0}\)with an unknown parameter\(\lambda \). That is, let the\({\mu _i}\)’s be independent with\({\mu _i}\)having the normal distribution with mean ψ and precision\(\,\lambda {T_i}\)conditional on\(\psi \),\(\lambda \) and\({T_1},\,....{T_P}\). Let\(\lambda \)have the gamma distribution with parameters\({\gamma _0}\)and\(\,{\delta _0}\), and let\(\lambda \)be independent of ψ and\({T_1},\,....{T_P}\). The remaining parameters have the prior distributions stated in Example 12.5.6.

a. Write the product of the likelihood and the prior as a function of the parameters\({\mu _i},...,{\mu _P}\,\), \({T_1},\,....{T_P}\)ψ, and\(\lambda \).

b. Find the conditional distributions of each parameter given all of the others. Hint: For all the parameters besides\(\lambda \), the distributions should be almost identical to those given in Example 12.5.6. It wherever\({\lambda _0}\)appears, of course, something will have to change.

c. Use a prior distribution in which α0 = 1, β0 = 0.1, u0 = 0.001, γ0 = δ0 = 1, and \({\psi _0}\)= 170. Fit the model to the hot dog calorie data from Example 11.6.2. Compute the posterior means of the four μi’s and 1/τi’s.

Short answer

1. The product of the two functions uses parameters given in the exercise and distribution \(\lambda \).
2. The conditional distribution of \(\lambda \) all other parameters is gamma distribution.
3. The estimated posterior means of \({\mu _i}\) \(i = 1,2,3,4\) are, respectively, 157 158.6, 118.9, and 160.4.
4. The estimated posterior means for the \(1/{T_i}\) \(i = 1,2,3,4\) are, respectively, 487, 598.9, 479.44, and 548.1.

Step-by-step solution

Normal distribution and gamma distribution

Recall what is stated in the mentioned example and the normal and gamma distribution probability density functions. Then, the required product is a product of

\(\exp \left\{ { - \frac{{{u_0}{{\left( {\psi - {\psi _0}} \right)}^2}}}{2} - \sum\limits_{i = 1}^p {{T_i}} \left( {{\beta _0} + \frac{{{n_i}{{\left( {{\mu _i} - \mathop {{y_i}}\limits^\_ } \right)}^2} + {w_i} + \lambda {{\left( {{\mu _i} - \psi } \right)}^2}}}{2}} \right)} \right\}\)

and the following expression

\({\lambda ^{p/2 + {\gamma _0} - 1}}\exp \left( { - \lambda {\delta _0}} \right)\,\,\mathop \Pi \limits_{i = 1}^p T_i^{{\alpha _0} + \left( {{n_i} + 1} \right)/2 - 1},\)

and \({w_i}\) is given in the same way as in the example

\({w_i} = \sum\limits_{j = 1}^{{n_i}} {{{\left( {{y_{ij}} - \mathop {{y_i}}\limits^\_ } \right)}^2}} ,\,\,\,\,i = 1,2,...,p.\)

The second term corresponds to the prior distribution.

Conditional distribution 

As mentioned in the hint, the conditional distributions stay practically the same as in the example.

If they look at a product as a function of \({T_i}\), then you get a gamma distribution with parameters

\({\alpha _0} + \frac{{{n_i} + 1}}{2}\,\) and \(\,\,{\beta _0} + \frac{{{n_i}{{\left( {{\mu _i} - \mathop {{y_i}}\limits^\_ } \right)}^2} + {w_i} + \lambda {{\left( {{\mu _i} - \psi } \right)}^2}}}{2}\)

Next, as a function of \(\psi \)by writing it in a form widely known, one gets a normal distribution with parameters

\(\frac{{{u_0}{\psi _0} + \lambda \sum\nolimits_{i = 1}^p {\,\,} {\mu _i}}}{{{u_0} + \lambda \sum\nolimits_{i = 1}^p {{T_i}\,} }}\,\) and \({u_0} + \lambda \sum\limits_{i = 1}^p {{T_i}} \,.\)

Before commenting on the distribution of given all others, notice that the product above looks like a probability density function of a normal distribution when it is a function of for the same reason. The mean and precision are given with

\(\frac{{{n_i}\mathop {{y_i}}\limits^\_ + \lambda \psi }}{{{n_i} + {\lambda _0}}}\) and \(\,{u_0}{T_i}\,\left( {{n_i} - \lambda } \right).\)

Finally, if the product is seen as a function of \(\lambda \), it approximately probability density function of gamma distribution with parameters

\(\frac{p}{2} + {\gamma _0}\,\) and \({\delta _0} + \frac{1}{2}\sum\limits_{i = 1}^p {{T_i}} {\left( {{\mu _i} - \psi } \right)^2}.\)

Estimated posterior 

The parameters that should be used are given in the exercise, and the others are

\({n_1} = 20\) for beef,

\({n_2} = 17\) for meat,

\({n_3} = 17\) for poultry,

\({n_4} = 9\) for specialty.

Means \({\mu _i}\) \(i = 1,2,3,4\)correspond to the indices of \({n_i}\), \(i = 1,2,3,4\)

The code used for this simulation uses N = 20000 Markov chains with I = 100000 iterations/steps. It means that there are a total of I = 100000 parameter vector from which the result is obtained. Note that the given code should be changed as the initial parameters are different. The estimated posterior means of \({\mu _i}\), \(i = 1,2,3,4\) are, respectively, 157, 158.6, 118.9, and 160.4. Similarly, the estimated posterior means for the \(1/{T_i}\) \(i = 1,2,3,4\) are respectively, 487 598.9, 479.44, and 548.1

Here, the result of part(a), part(b) and part(c)

1. The product of the two functions uses parameters given in the exercise and distribution \(\lambda \).
2. The conditional distribution of \(\lambda \) all other parameters is gamma distribution.
3. The estimated posterior means of \({\mu _i}\) \(i = 1,2,3,4\) are, respectively, 157 158.6, 118.9, and 160.4.
4. The estimated posterior means for the \(1/{T_i}\) \(i = 1,2,3,4\) are, respectively, 487 598.9, 479.44, and 548.1.