Question

Show that the multivariate normal distribution \(N_{p}(\mu, \Omega)\) is a group transformation model under the map \(Y \mapsto a+B Y\), where \(a\) is a \(p \times 1\) vector and \(B\) an invertible \(p \times p\) matrix. Given a random sample \(Y_{1}, \ldots, Y_{n}\) from this distribution, show that $$ \bar{Y}=n^{-1} \sum_{j=1}^{n} Y_{j}, \quad \sum_{j=1}^{n}\left(Y_{j}-\bar{Y}\right)\left(Y_{j}-\bar{Y}\right)^{\mathrm{T}} $$ is a minimal sufficient statistic for \(\mu\) and \(\Omega\), and give equivariant estimators of them. Use these estimators to find the maximal invariant.

Short answer

The multivariate normal distribution is a group transformation model, with sufficient statistics and equivariant estimators \( \bar{Y} \) and the sample covariance matrix. The maximal invariant involves the transformed Mahalanobis distance.

Step-by-step solution

Define the Transformation

Consider the linear transformation \( Y \mapsto a + BY \), where \( a \) is a \( p \times 1 \) vector and \( B \) is an invertible \( p \times p \) matrix. This transformation is applied to a multivariate normal vector \( Y \sim N_p(\mu, \Omega) \).

Show the Distribution Under Transformation

The transformed vector \( a + BY \) has a new mean and covariance. The mean becomes \( a + B\mu \) and the covariance matrix is \( B\Omega B^T \), confirming that a linear transformation of a multivariate normal distribution results in another multivariate normal distribution.

Define Sufficient Statistics for the Original Model

The sample mean, \( \bar{Y} = n^{-1} \sum_{j=1}^{n} Y_j \), and the sample covariance matrix, \( \sum_{j=1}^{n} (Y_j - \bar{Y})(Y_j - \bar{Y})^T \), are used as sufficient statistics for the parameters \( \mu \) and \( \Omega \) respectively in the original model.

Consider the Equivariance Under Transformation

Due to the properties of matrix operations and transformations, the statistics \( \bar{Y} \) and the sample covariance remain equivariant under the transformation \( Y \mapsto a + BY \), meaning that they transform in the same manner as the data.

Minimal Sufficiency Verification

Verify that \( (\bar{Y}, S) \) is minimal sufficient by using the factorization theorem, which states that the likelihood can be factored into functions of the statistics and the parameters, confirming minimal sufficiency.

Determine Equivariant Estimators

The equivariant estimator for \( \mu \) remains \( \bar{Y} \), transformed appropriately as \( a + B\bar{Y} \) for consitency with the map. For \( \Omega \), the estimator is the sample covariance matrix scaled appropriately by the transformation \( B \).

Find the Maximal Invariant

The maximal invariant can be found by constructing the statistic that remains unchanged under the group transformation. For a location-scale family, the maximal invariant typically involves the Mahalanobis distance, derived from the differences \( Y_j - \bar{Y} \), excitingly satisfying the group-invariance.

Key concepts

Group Transformation Model

A Group Transformation Model is a framework in multivariate statistics where transformations are applied to random variables within a group structure. In the context of the multivariate normal distribution, we explore this idea using an affine transformation:

• Represented by the equation \( Y \mapsto a + BY \), where \( a \) is a vector and \( B \) is an invertible matrix.
• This transformation accounts for changes in both location and scale, which is inherent to the properties of multivariate distributions.

When applying this transformation to a multivariate normal distribution, several key aspects arise:

• The mean of the distribution transforms from \( \mu \) to \( a + B\mu \).
• The covariance matrix, initially \( \Omega \), transforms to the new matrix \( B\Omega B^T \).

This shows that the transformed random variable remains multivariate normal. It is essential in understanding how complex datasets might transform under linear operations, maintaining the core properties that define the distribution.

Sufficient Statistic

In statistical modeling, a sufficient statistic contains all the necessary information about a parameter based on a sample. For the multivariate normal distribution, the sufficient statistics are the sample mean and the sample covariance matrix.

• The sample mean is given by \( \bar{Y} = n^{-1} \sum_{j=1}^{n} Y_j \).
• The sample covariance matrix is \( \sum_{j=1}^{n} (Y_j - \bar{Y})(Y_j - \bar{Y})^T \).

These statistics serve as complete summaries of the data concerning the parameters \( \mu \) and \( \Omega \). That means:

• No additional sample information could alter the inference about these parameters, given the sample mean and covariance are known.
• They simplify the data without losing relevant information about the population parameters, crucial for estimation and hypothesis testing.

Equivariant Estimators

Equivariant estimators are those that preserve or respect the transformation structure applied to the data. This is vital in affirming the consistency and unbiased nature of the estimation.

• For the mean \( \mu \), the estimator is \( \bar{Y} \), which transforms as \( a + B\bar{Y} \).
• For the covariance \( \Omega \), the estimator is the sample covariance matrix, scaled by the transformation \( B \).

These estimators adjust according to the group transformation, retaining their validity under variable transformations. As a result:

• They are both unbiased and exhibit minimal variance under the group operations.
• This quality makes them particularly reliable for making predictions when the data undergoes transformations typical in practices like data normalization or scaling.

Factorization Theorem

The Factorization Theorem provides a tool to determine minimal sufficiency, by allowing the probability (likelihood) to be expressed in terms of functions of statistics and parameters.

• The theorem states that a statistic is sufficient if the likelihood function can be decomposed into a product, where one factor depends on the data only through the statistic itself.
• This helps in confirming whether given statistics are not just sufficient but minimal, making them efficient in parameter estimation.

In the relation to multivariate normal distributions:

• The statistics \( \bar{Y} \) and the sample covariance matrix suffice for \( \mu \) and \( \Omega \), reducing the complexity of handling multivariate observations by summarizing key information.
• This decomposition ensures the estimator's computation and analysis remain computationally feasible, even with high-dimensional data structures.