Question

Consider the following algorithm: 1\. Generate \(U\) and \(V\) independent uniform \((-1,1)\) random variables. 2\. Set \(W=U^{2}+V^{2}\). 3\. If \(W>1\) go to step 1 . 4\. Set \(Z=\sqrt{(-2 \log W) / W}\) and let \(X_{1}=U Z\) and \(X_{2}=V Z\). Show that the random variables \(X_{1}\) and \(X_{2}\) are iid with a common \(N(0,1)\) distribution. This algorithm was proposed by Marsaglia and Bray (1964).

Short answer

By deriving the joint density of \(X_{1}\), \(X_{2}\), and \(Z\), and then calculating the marginal densities of \(X_{1}\) and \(X_{2}\), we find that the joint intensity can be expressed as the product of two marginal densities. This indicates that \(X_{1}\) and \(X_{2}\) are independent. Furthermore, it can be shown that both marginal densities take the form of a standard normal distribution, verifying that the algorithm indeed produces independent and identically distributed random variables according to the \(N(0,1)\) distribution.

Step-by-step solution

Definition of a standard normal variable

Recall that a random variable \(X\) follows a standard normal distribution \(N(0,1)\) if its probability density function is given by \(f(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}\). It is also the transformation of a uniform random variable to a standard normal variable that's focused on here. It's clear that both \(X_{1}\) and \(X_{2}\) are produced with the help of uniform random variables \(U\) and \(V\).

Calculation of joint density function

Given \(U\) and \(V\) are uncorrelated uniform \((-1,1)\) random variables, they have a joint density function \(f_{U,V}(u,v) = 1/4\) for \(-1<u,v<1\), 0 elsewhere. Therefore, we can introduce the transformation \(X_{1} = UZ\) and \(X_{2} = VZ\), where \(Z = \sqrt{-2 \log W / W}\) and \(W = U^2 + V^2\). The joint density of \(X_{1}\), \(X_{2}\), and \(Z\) can be calculated by the Jacobian transformation method. Specifically, the new variables \(W\), \(X_{1}\), and \(X_{2}\) are functions of \(U\), \(V\), and \(Z\).

Calculation of marginal densities

Calculate the joint density \(f_{X_1,X_2,Z}(x_1,x_2,z)\). Notice that \(z = \sqrt{-2 \log w / w}\), which implies that \(z > 0\) when \(w > 0\). Thus, the joint density is 0 when \(w = 0\) or when \(z \leq 0\). Then find the marginal densities \(f_{X_1}(x_1)\) and \(f_{X_2}(x_2)\) by integrating out \(X_2\) and \(X_1\) respectively.

Show that \(X_{1}\) and \(X_{2}\) are independent and follow the standard normal distribution

Independence of \(X_{1}\) and \(X_{2}\) is established if and only if the joint density function can be written as a product of two marginal density functions. Look for the joint probability density function \(f_{X_1,X_2}(x_1, x_2) = f_{X_1}(x_1) f_{X_2}(x_2)\). It should match the form of the standard normal distribution, which means \(X_{1}\) and \(X_{2}\) independently follow \(N(0,1)\).

Key concepts

Marsaglia-Bray algorithm

When exploring the realms of probability and statistics, the transformation of variables is a common occurrence, particularly when one desires to simulate random variables that follow a specific distribution. One innovative approach to generating normal random variables is the Marsaglia-Bray algorithm, a method devised in the mid-1960s. This technique circumvents the need for complex integration or summation processes, which can be computationally intensive.

The algorithm is relatively straightforward and begins with the generation of two uniform random variables, denoted as U and V, whose values range from -1 to 1. These variables are used to compute another variable W, defined as U squared plus V squared. If W exceeds 1, the process restarts, indicating a rejection-based sampling method. Once a suitable W is found, it is used to calculate Z, which scales both U and V to produce the normal random variables X₁ and X₂.

What makes the algorithm efficient is that it directly utilizes the properties of the uniform distribution and the geometry of circles to produce variables from the standard normal distribution without explicitly using normal distribution functions. The repetition of step 3 in the algorithm ensures that the values of W are always less than or equal to 1, correlating to the fact that the points (U, V) lie within a unit circle. This directly correlates with the radial symmetry of the standard normal distribution in two dimensions.

Independent and identically distributed (iid)

When studying the concept of independence in statistics, it is crucial to recognize its pivotal role in simplifying complex problems, such as in the instance of the Marsaglia-Bray algorithm. Independence suggests that the occurrence of one event does not influence the probability of another. In the algorithm, X₁ and X₂ demonstrate this property; knowing the value of X₁ doesn't provide any information about the value of X₂, and vice versa.

Alongside independence, the concept of identically distributed variables implies that all variables share the same probability distribution. The 'iid' condition in the Marsaglia-Bray algorithm infers that both X₁ and X₂ conform to the standard normal distribution, denoted by N(0,1). The importance of this property cannot be overstated as it accredits the algorithm with the ability to simulate standard normal variables reliably.

Verifying the 'iid' property for X₁ and X₂ involves demonstrating that the random variables are not only independent but also exhibit the same standard normal distribution. This entails proving the joint density function can be decomposed into the product of individual marginal density functions, each representing a standard normal distribution.

Jacobian transformation method

The interplay between random variables often necessitates transformation to shift from one distribution to another, and this is where the Jacobian transformation method comes into play. This methodology is indispensable, particularly within the Marsaglia-Bray algorithm, for transitioning from the uniform distribution of U and V to the standard normal distribution.

The crux of this transformation banks on the Jacobian, a determinant that arises during the change of variables in multivariate integrals. The purpose here is to connect the joint density function of one set of variables to another set through a deterministic mapping. In the context of our Marsaglia-Bray algorithm, the transformation utilizes U, V, and the geometrically derived variable Z to produce the new variables X₁ and X₂.

The Jacobian transformation effectively recalibrates the density functions, enabling us to demonstrate that X₁ and X₂ not only follow a normal distribution but also confirm their independence. This is achieved by showing that the joint density of X₁ and X₂ can be expressed as the product of two independent standard normal marginal densities. Such an insight is not just a theoretical novelty but a practical tool in statistical simulations.