Question

Consider the situation of the last exercise, but suppose we have the following two independent random samples: (1). \(X_{1}, X_{2}, \ldots, X_{n}\) is a random sample with the common pdf \(f_{X}(x)=\theta^{-1} e^{-x / \theta}\), for \(x>0\), zero elsewhere, and (2). \(Y_{1}, Y_{2}, \ldots, Y_{n}\) is a random sample with common pdf \(f_{Y}(y)=\tau e^{-\tau y}\), for \(y>0\), zero elsewhere. Assume that \(\tau=1 / \theta\) The last exercise suggests that, for some constant \(c, Z=c \bar{X} / \bar{Y}\) might be an unbiased estimator of \(\theta^{2}\). Find this constant \(c\) and the variance of \(Z\). Hint: Show that \(\bar{X} /\left(\theta^{2} \bar{Y}\right)\) has an \(F\) -distribution.

Short answer

The constant \(c = \theta^{2}\) and the variance of \(Z\) is \(\frac{1}{n}\) given \(n>2\).

Step-by-step solution

Recognize the Distribution of Random Variables

First, note that if \(X\) follows a Gamma distribution with parameters \(n\) and \(\theta\) (i.e., \(X\sim \text{Gamma}(n,\theta)\)), its pdf is \(f_{X}(x)=\theta^{-n}\frac{x^{n-1} e^{-x / \theta}}{(n-1)!}\) where \(x>0\). By comparing, we can observe that \(X_{1}, X_{2}, \ldots, X_{n}\) are all distributed according to the Gamma distribution with \(n=1\) and \(\theta = \theta\). Similarly, \(Y_{1}, Y_{2}, \ldots, Y_{n}\) are Gamma distributed with \(n=1\) and \(\theta = \frac{1}{\tau}\).

Proof of F-Distribution

To show that \(\bar{X} /\left(\theta^{2} \bar{Y}\right)\) has an \(F\)-distribution, we define the random variable \(W = \bar{X} /\left(\theta^{2} \bar{Y}\right)\). In an \(F\)-distribution with \(d_{1}\) and \(d_{2}\) degrees of freedom, \(F\) can be expressed as \(F=\frac{X / d_{1}}{Y / d_{2}}\) where \(X\) follows \(\text{Chi-square}(d_1)\) distribution and \(Y\) follows \(\text{Chi-Square}(d_2)\) distribution. Since summing \(n\) independent Gamma(1, \(\theta\)) variables produces a Gamma(\(n\), \(\theta\)) variable, and a Gamma(\(n\), \(1\)) variable is the same as a \(\text{Chi-square}(2n)\) variable, we have \(\sum_{i=1}^{n}X_{i}\) follows a \(\text{Chi-square}(2n)\) distribution and \(\sum_{i=1}^{n}Y_{i}\) follows a \(\text{Chi-square}(2n)\) distribution. Thus, \(W\) follows a \(F\)-distribution with \(2n\) and \(2n\) degrees of freedom.

Find the constant

Given that \(Z = c\frac{\bar{X}}{\bar{Y}}\) and it is an unbiased estimator of \(\theta^{2}\), use the property of \(F\)-distribution where the expected value \(E(F)\) is \(\frac{d_{2}}{d_{2}-2}\) when \(d_{2}>2\), we find that \(E(Z) = \theta^{2}\), which implies that \(E(c\frac{\bar{X}}{\bar{Y}}) = \theta^{2}\). This simplifies to \(cE(\frac{\bar{X}}{\theta \bar{Y}}) = \theta^{2}\), which, by referring back to \(W\), equals \(c E(W) = \theta^{2}\). Given \(E(W) = d_{2}\) when \(d_{2}>2\) and the denominator degrees of freedom for \(W\) is \(2n = d_{2}\), we find that \(c = \theta^{2}\).

Calculate Variance of Z

To calculate \(Var(Z)\), first, we use the property of \(F\)-distribution where the variance \(Var(F)\) is \(\frac{2d_{2}^{2} (d_{1}+d_{2}-2)}{d_{1}(d_{2}-2)^{2}(d_{2}-4)}\) for \(d_{2}>4\). Since the pdf of \(Z\) is an \(F\) distribution with \(d_{1}=d_{2}=2n\), we substitute these values into the formula for the variance of an \(F\)-distribution and simplify to obtain \(Var(Z) = \frac{2 * (2n)^2 * (4n-2)}{2n * (4n - 4)^2 * (2n - 4)} = \frac{1}{n}\), given \(n>2\).

Key concepts

Gamma Distribution

The Gamma distribution plays a critical role in statistical theories and applications. It is primarily used to model the wait time until the k-th occurrence of an event in a Poisson process. In our exercise, each random variable, such as \(X_1, X_2, \text{and} Y_1, Y_2\), follows a Gamma distribution with shape parameter \(n\) and scale parameter \(\theta\).

For a given shape parameter \(n\) and scale parameter \(\theta\), the probability density function (pdf) of the Gamma distribution is expressed as: \[ f_{X}(x)=\theta^{-n}\frac{x^{n-1} e^{-x / \theta}}{(n-1)!} \] where \(x>0\). This formula shows how often we might expect different outcomes, with \(\theta\) affecting the distribution's spread and \(n\) its shape. In simple terms, for our samples in the exercise, the occurrence of each \(X_i\) and \(Y_i\) can be described mathematically by this formula, with \(n=1\) stipulating that we are dealing with the first occurrence of an event in a Poisson process.

F-Distribution

The F-distribution is another important concept in statistics, especially when comparing variances or testing hypotheses about means. It is the distribution of the ratio of two independent chi-squared variables divided by their respective degrees of freedom.

In the context of our exercise, we encountered a situation where we required finding out if a specific random variable had an F-distribution. The random variable \(W\), which is \(\bar{X} / (\theta^{2} \bar{Y})\), is shown to follow an F-distribution. An interesting property of the F-distribution is that it is naturally related to the Gamma distribution because a chi-squared random variable is a special case of the Gamma distribution.

To put it simply, when we take the gamma-distributed samples and create ratios similar to the one described in the problem, we might reveal an F-distribution. This explains why \(W\) follows an F-distribution with \(2n\) and \(2n\) degrees of freedom—each chi-squared variable being a sum of \(n\) gamma-distributed variables.

Unbiased Estimator

An unbiased estimator is a statistical tool that accurately targets the expected value of the population parameter it aims to estimate. This means that, on average, it neither overestimates nor underestimates the parameter.

In our exercise, the quest is to find a constant \(c\) such that \(Z=c \bar{X} / \bar{Y}\) becomes an unbiased estimator for \(\theta^{2}\), essentially being correct on average across many samples. Based on the properties of the F-distribution we worked with, we proved that when we properly tweak the random variable using the right \(c\), the expected value of \(Z\) (i.e., \(E(Z)\)) corresponds to \(\theta^{2}\).

By solving for \(c\), we can construct an estimator that works perfectly in theory, meaning if we could take an infinite number of samples, our estimator would give us the exact value of \(\theta^{2}\). However, in real-world applications, an unbiased estimator is powerful even with a finite number of samples, as it ensures no systematic bias towards over- or under-prediction. In summary, the constancy \(c\) when determined correctly, empowers us with an invaluable tool for making accurate and reliable inferences in statistics.