Question

Let \(Y\) have the \(p\)-variate multivariate normal distribution with mean vector \(\mu\) and covariance matrix \(\Omega\). Partition \(Y^{\mathrm{T}}\) as \(\left(Y_{1}^{\mathrm{T}}, Y_{2}^{\mathrm{T}}\right)\), where \(Y_{1}\) has dimension \(q \times 1\) and \(Y_{2}\), has dimension \(r \times 1\), and partition \(\mu\) and \(\Omega\) conformably. Find the conditional distribution of \(Y_{1}\) given that \(Y_{2}=y_{2}\) direct from the probability density functions of \(Y\) and \(Y_{2}\).

Short answer

The conditional distribution of \( Y_1 \) given \( Y_2 = y_2 \) is multivariate normal with mean \( \mu_1 + \Omega_{12}\Omega_{22}^{-1}(y_2 - \mu_2) \) and covariance \( \Omega_{11} - \Omega_{12}\Omega_{22}^{-1}\Omega_{21} \).

Step-by-step solution

Define the distribution of Y

The random vector \( Y \) is partitioned as \( \begin{pmatrix} Y_1 \ Y_2 \end{pmatrix} \). \( Y \) follows a multivariate normal distribution with mean vector \( \mu \) and covariance matrix \( \Omega \). We partition \( \mu \) and \( \Omega \) conformably as follows: \( \mu = \begin{pmatrix} \mu_1 \ \mu_2 \end{pmatrix} \) and \( \Omega = \begin{pmatrix} \Omega_{11} & \Omega_{12} \ \Omega_{21} & \Omega_{22} \end{pmatrix} \).

Write the joint distribution of Y1 and Y2

Using the properties of the multivariate normal distribution, the joint distribution of \( Y_1 \) and \( Y_2 \) can be expressed as:\[\begin{pmatrix} Y_1 \ Y_2 \end{pmatrix} \sim \mathcal{N}\left( \begin{pmatrix} \mu_1 \ \mu_2 \end{pmatrix}, \begin{pmatrix} \Omega_{11} & \Omega_{12} \ \Omega_{21} & \Omega_{22} \end{pmatrix} \right)\]

Find the distribution of Y2

The distribution of \( Y_2 \) is obtained by projecting the joint distribution on \( Y_2 \). Thus, \( Y_2 \sim \mathcal{N}(\mu_2, \Omega_{22}) \) since \( Y_2 \) is part of the multivariate normal distribution with mean \( \mu_2 \) and covariance \( \Omega_{22} \).

Derive the conditional distribution of Y1 given Y2=y2

The conditional distribution of \( Y_1 \) given \( Y_2 = y_2 \) is multivariate normal with mean and covariance given by:\[Y_1 | Y_2 = y_2 \sim \mathcal{N}(\mu_{1 \,|\, 2}, \Omega_{1 \,|\, 2})\]where \( \mu_{1 \,|\, 2} = \mu_1 + \Omega_{12}\Omega_{22}^{-1}(y_2 - \mu_2) \) and \( \Omega_{1 \,|\, 2} = \Omega_{11} - \Omega_{12}\Omega_{22}^{-1}\Omega_{21} \).

Express the final conditional distribution

The final conditional distribution is expressed neatly using the expressions for the mean and covariance derived above:\[Y_1 | Y_2 = y_2 \sim \mathcal{N}\left( \mu_1 + \Omega_{12}\Omega_{22}^{-1}(y_2 - \mu_2), \Omega_{11} - \Omega_{12}\Omega_{22}^{-1}\Omega_{21} \right)\]

Key concepts

Mean Vector

In the context of a multivariate normal distribution, the mean vector is a critical component. It essentially tells us the "center" of the data in a multivariate space. Imagine a multidimensional graph where each axis is one variable from your data. The mean vector \( \mu \) is a point in this space that represents the average of all the points.For example, if your data has two variables, the mean vector might look something like \( \begin{pmatrix} \mu_1 \ \mu_2 \end{pmatrix} \), where \( \mu_1 \) is the mean of the first variable and \( \mu_2 \) is the mean of the second variable. This concept is important because it helps in determining how data is distributed around this center point.

Covariance Matrix

The covariance matrix, denoted as \( \Omega \) in our multivariate normal distribution, provides vital insights into the relationships between the different variables in your data. Each element in this matrix represents covariances between pairs of variables, essentially measuring how changes in one variable are associated with changes in another.The diagonal elements of this matrix represent variances of individual variables. Since variance is the square of standard deviation, it provides a measure of the spread of the data for each variable. The off-diagonal elements represent the covariance between different variables, indicating if changes in one variable are linked to changes in another. When the covariance is positive, it indicates that as one variable increases, the other tends to increase, too; if it's negative, as one increases, the other tends to decrease. Understanding the covariance matrix is crucial for grasping the geometry and orientation of the data cloud in a multivariate space.

Conditional Distribution

Conditional distributions allow us to understand how one part of our variable set behaves given some information about another part. In the multivariate normal scenario, we're calculating the distribution for one group of variables given specific values for another group.Simply put, if \( Y_2 \) is one set of variables and you know its value, you can determine how \( Y_1 \) behaves. The conditional distribution \( Y_1 | Y_2 = y_2 \) is still a multivariate normal distribution but with a new mean vector and covariance matrix. The new mean vector, represented as \( \mu_{1 | 2} = \mu_1 + \Omega_{12}\Omega_{22}^{-1}(y_2 - \mu_2) \), is adjusted based on the known value of \( Y_2 \). Similarly, the new covariance matrix \( \Omega_{1 | 2} \) becomes \( \Omega_{11} - \Omega_{12} \Omega_{22}^{-1} \Omega_{21} \), which accounts for this conditioning.

Probability Density Function

The probability density function (pdf) is a fundamental concept in probability and statistics, especially in the context of continuous random variables like those in a multivariate normal distribution. The pdf describes the likelihood of a random variable taking on a particular value. In the case of the multivariate normal distribution, the pdf is a bit complex due to the involvement of multiple variables. It involves an exponential function of the variables’ distances from their mean vector, scaled by the covariance matrix. The pdf is crucial for defining the distribution and finding probabilities over ranges of values for the variables. Understanding the pdf provides a way to compute the probability of observing a set of outcomes within a particular range. It also helps to express the overall shape and spread of the data distribution, and importantly, how similar or different all these data observations are from the mean vector.