Question

Let \(\mathbf{X}_{1}, \mathbf{X}_{2}, \ldots, \mathbf{X}_{n}\) be a random sample from a multivariate normal normal distribution with mean vector \(\boldsymbol{\mu}=\left(\mu_{1}, \mu_{2}, \ldots, \mu_{k}\right)^{\prime}\) and known positive definite covariance matrix \(\mathbf{\Sigma}\). Let \(\overline{\mathbf{X}}\) be the mean vector of the random sample. Suppose that \(\mu\) has a prior multivariate normal distribution with mean \(\boldsymbol{\mu}_{0}\) and positive definite covariance matrix \(\boldsymbol{\Sigma}_{0}\). Find the posterior distribution of \(\mu\), given \(\overline{\mathbf{X}}=\overline{\mathbf{x}}\). Then find the Bayes estimate \(E(\boldsymbol{\mu} \mid \overline{\mathbf{X}}=\overline{\mathbf{x}})\).

Short answer

Posterior distribution of \( \boldsymbol{\mu}| \overline{\mathbf{X}} = \overline{\mathbf{x}} \) is multivariate normal with mean \( ( \Sigma^{-1} + n \Sigma_0^{-1} ) ^{-1} ( \Sigma^{-1} \boldsymbol{\mu}_{0} + n \Sigma_0^{-1} \overline{\mathbf{x}}) \) and covariance \( ( \Sigma^{-1} + n \Sigma_0^{-1} ) ^{-1} \). The Bayes estimate \( E(\boldsymbol{\mu}| \overline{\mathbf{X}} = \overline{\mathbf{x}}) \) is \( ( \Sigma^{-1} + n \Sigma_0^{-1} ) ^{-1} ( \Sigma^{-1} \boldsymbol{\mu}_{0} + n \Sigma_0^{-1} \overline{\mathbf{x}}) \).

Step-by-step solution

Unerstand the given parameters

The problem provides us with a number of parameters. The multivariate normal distribution's mean vector is \( \boldsymbol{\mu} = \left(\mu_{1}, \mu_{2}, ..., \mu_{k}\right)^{\prime}\) and it has a known covariance matrix \( \mathbf{\Sigma} \). We are also told that \( \mu \) has a prior multivariate normal distribution with the mean \( \boldsymbol{\mu}_{0}\) and a covariance matrix \( \boldsymbol{\Sigma}_{0} \). We need to find the posterior distribution of \( \mu \), given \( \overline{\mathbf{X}} = \overline{\mathbf{x}} \), and then find the Bayes estimate \(E(\boldsymbol{\mu}| \overline{\mathbf{X}} = \overline{\mathbf{x}})\).

Find the posterior distribution of µ

Using properties of normal distribution and Bayes' theorem, the posterior distribution of \( \boldsymbol{\mu} \), given \( \overline{\mathbf{X}} = \overline{\mathbf{x}} \) (denoted as \( \boldsymbol{\mu}| \overline{\mathbf{X}} = \overline{\mathbf{x}} \)) is multivariate normal with mean \( E[\boldsymbol{\mu}|\overline{\mathbf{X}} = \overline{\mathbf{x}}] = ( \Sigma^{-1} + n \Sigma_0^{-1} ) ^{-1} ( \Sigma^{-1} \boldsymbol{\mu}_{0} + n \Sigma_0^{-1} \overline{\mathbf{x}}) \) and covariance \( ( \Sigma^{-1} + n \Sigma_0^{-1} ) ^{-1} \).

Find the Bayes Estimate

The Bayes estimate \( E(\boldsymbol{\mu}| \overline{\mathbf{X}} = \overline{\mathbf{x}}) \) is the mean of the posterior distribution. Using the result from the previous step, we can conclude that the Bayes estimate is \( ( \Sigma^{-1} + n \Sigma_0^{-1} ) ^{-1} ( \Sigma^{-1} \boldsymbol{\mu}_{0} + n \Sigma_0^{-1} \overline{\mathbf{x}}) \).