Question

Let \(X_{1}, \ldots, X_{n}\) be independent random variables having a common distribution function that is specified up to an unknown parameter \(\theta\). Let \(T=T(\mathrm{X})\) be a function of the data \(\mathrm{X}=\left(X_{1}, \ldots, X_{n}\right) .\) If the conditional distribution of \(X_{1}, \ldots, X_{n}\) given \(T(\mathrm{X})\) does not depend on \(\theta\) then \(T(\mathrm{X})\) is said to be a sufficient statistic for \(\theta .\) In the following cases, show that \(T(\mathbf{X})=\sum_{i=1}^{n} X_{i}\) is a sufficient statistic for \(\theta\). (a) The \(X_{i}\) are normal with mean \(\theta\) and variance \(1 .\) (b) The density of \(X_{i}\) is \(f(x)=\theta e^{-\theta x}, x>0\). (c) The mass function of \(X_{i}\) is \(p(x)=\theta^{x}(1-\theta)^{1-x}, x=0,1,0<\theta<1\). (d) The \(X_{i}\) are Poisson random variables with mean \(\theta\).

Short answer

In summary, we have shown that in all four cases, \(T(\mathbf{X}) = \sum_{i=1}^n X_i\) is a sufficient statistic for the parameter \(\theta\) using the Factorization Theorem. The specific factorizations for each case are: (a) \(f(x_1, x_2, ... x_n ; \theta) = e^{\theta T(\mathbf{X}) - \theta^2n/2} \cdot \frac{1}{(2\pi)^{n/2}}e^{-\sum_{i=1}^n X_i^2/2}\) (b) \(f(x_1, x_2, ... x_n ; \theta) = \theta^n e^{-\theta T(\mathbf{X})} \cdot 1\) (c) \(f(x_1, x_2, ... x_n ; \theta) = \theta^{T(\mathbf{X})}(1-\theta)^{n-T(\mathbf{X})} \cdot 1\) (d) \(f(x_1, x_2, ... x_n ; \theta) = e^{-n\theta}\theta^{T(\mathbf{X})} \cdot \prod_{i=1}^n \frac{1}{x_i!}\)

Step-by-step solution

a) Normal Distribution

In this case, the \(X_i\) are normally distributed with mean \(\theta\) and variance \(1.\) The joint probability density function of the data can be written as: \[f(x_1, x_2, ... x_n ; \theta) = \prod_{i=1}^n \frac{1}{\sqrt{2\pi}} e^{-(x_i - \theta)^2/2}\] Now, we'll try to find the factorization of this function that separates the data and the statistic. Let \(T(\mathbf{X}) = \sum_{i=1}^n X_i\), so we can rewrite the function as: \[f(x_1, x_2, ... x_n ; \theta) = \frac{1}{(2\pi)^{n/2}} e^{\theta T(\mathbf{X}) - \theta^2n/2 - \sum_{i=1}^n X_i^2/2}\] Comparing the given function with the factorization result form, we can see that it can be factored as: \[ g(T(\mathbf{X});\theta) = e^{\theta T(\mathbf{X}) - \theta^2n/2} \qquad \text{and} \qquad h(\mathbf{X}) = \frac{1}{(2\pi)^{n/2}}e^{-\sum_{i=1}^n X_i^2/2}\] As we have found a factorization of the density function of the form \(f(x_1, x_2, ... x_n ; \theta) = g(T(\mathbf{X});\theta) h(\mathbf{X})\), the factorization theorem states that \(T(\mathbf{X}) = \sum_{i=1}^n X_i\) is a sufficient statistic for \(\theta\) in this normal case.

b) Exponential Distribution

In this case, the random variables have a density function, \(f(x)=\theta e^{-\theta x}, x>0\). Therefore, the joint probability density function can be written as: \[f(x_1, x_2, ... x_n ; \theta) = \prod_{i=1}^n \theta e^{-\theta x_i}\] which simplifies to: \[f(x_1, x_2, ... x_n ; \theta) = \theta^n e^{-\theta\sum_{i=1}^n x_i}\] This function can be factored as: \[ g(T(\mathbf{X});\theta) = \theta^n e^{-\theta T(\mathbf{X})} \qquad \text{and} \qquad h(\mathbf{X}) = 1\] where \(T(\mathbf{X}) = \sum_{i=1}^n X_i\). As the function can be factored in the required form, the factorization theorem implies that \(T(\mathbf{X}) = \sum_{i=1}^n X_i\) is a sufficient statistic for \(\theta\) in this case.

c) Bernoulli Distribution

In this case, the mass function for \(X_i\) is \(p(x)=\theta^{x}(1-\theta)^{1-x}, x=0,1,0<\theta<1\). The joint probability mass function can be written as: \[f(x_1, x_2, ... x_n ; \theta) = \prod_{i=1}^n \theta^{x_i}(1-\theta)^{1-x_i}\] which simplifies to: \[f(x_1, x_2, ... x_n ; \theta) = \theta^{\sum_{i=1}^n x_i} (1-\theta)^{n-\sum_{i=1}^n x_i}\] This function can be factored as: \[g(T(\mathbf{X});\theta) = \theta^{T(\mathbf{X})}(1-\theta)^{n-T(\mathbf{X})} \qquad \text{and} \qquad h(\mathbf{X}) = 1\] where \(T(\mathbf{X}) = \sum_{i=1}^n X_i\). As the function can be factored in the required form, the factorization theorem implies that \(T(\mathbf{X}) = \sum_{i=1}^n X_i\) is a sufficient statistic for \(\theta\) in this case.

d) Poisson Distribution

In this case, the \(X_i\) are Poisson distributed with mean \(\theta\), and the probability mass function for \(X_i\) is: \[p(x_i) = \frac{e^{-\theta}\theta^{x_i}}{x_i!}\] The joint probability mass function can be written as: \[f(x_1, x_2, ... x_n ; \theta) = \prod_{i=1}^n \frac{e^{-\theta}\theta^{x_i}}{x_i!}\] which simplifies to: \[f(x_1, x_2, ... x_n ; \theta) = e^{-n\theta}\theta^{\sum_{i=1}^n x_i}\prod_{i=1}^n \frac{1}{x_i!}\] The function can be factored as: \[g(T(\mathbf{X});\theta) = e^{-n\theta}\theta^{T(\mathbf{X})} \qquad \text{and} \qquad h(\mathbf{X}) = \prod_{i=1}^n \frac{1}{x_i!}\] where \(T(\mathbf{X}) = \sum_{i=1}^n X_i\). As the function can be factored in the required form, the factorization theorem implies that \(T(\mathbf{X}) = \sum_{i=1}^n X_i\) is a sufficient statistic for \(\theta\) in this case. In summary, we have shown that in all four cases, \(T(\mathbf{X}) = \sum_{i=1}^n X_i\) is a sufficient statistic for the parameter \(θ\) using the Factorization Theorem.