Chapter 10

Let \(\Omega\) consist of four points, each with probability \(\frac{1}{4}\). Find three events that are pairwise independent but not independent. Generalize.

(The Moment Convergence Theorem) Let \(X_{1}, X_{2}, \ldots, X\) be random variables such that \(P_{X_{n}} \rightarrow P_{X}\) vaguely and \(\sup _{n} E\left(\left|X_{n}\right|^{r}\right)<\infty\), where \(r>0\). Then \(E\left(\left|X_{n}\right|^{s}\right) \rightarrow E\left(|X|^{s}\right)\) for all \(s \in(0, r)\), and if also \(s \in \mathbb{N}\), then \(E\left(X_{n}^{s}\right) \rightarrow E\left(X^{s}\right)\). (By Chebyshev's inequality, if \(\epsilon>0\), there exists \(a>0\) such that \(P\left(\left|X_{n}\right|>a\right)<\epsilon\) for all \(n\). Consider \(\int \phi(t)|t|^{s} d P_{X_{n}}(t)\) and \(\int[1-\phi(t)]|t|^{s} d P_{X_{n}}(t)\) where \(\phi \in C_{c}(\mathbb{R})\) and \(\phi(t)=1\) for \(|t| \leq a\).)

If \(\left\\{a_{n}\right\\} \subset C\) and \(\lim a_{n}=a\), then \(\lim n^{-1} \sum_{1}^{n} a_{j}=a\).

(Shannon's Theorem) Let \(\left\\{X_{i}\right\\}\) be a sequence of independent random variables on the sample space \(\Omega\) having the common distribution \(\lambda=\sum_{1}^{r} p_{j} \delta_{j}\) where \(0<\) \(p_{j}<1, \sum_{1}^{r} p_{j}=1\), and \(\delta_{j}\) is the point mass at \(j .\) Define random variables \(Y_{1}, Y_{2}, \ldots\) on \(\Omega\) by $$ Y_{n}(\omega)=P\left(\left\\{\omega^{\prime}: X_{i}\left(\omega^{\prime}\right)=X_{i}(\omega) \text { for } 1 \leq i \leq n\right\\}\right) . $$ a. \(Y_{n}=\prod_{1}^{n} p_{X_{i}} .\) (The notation is peculiar but correct: \(X_{i}(\cdot) \in\\{1, \ldots, r\\}\) a.s., so \(p X_{i}\) is well-defined a.s.) b. \(\lim _{n \rightarrow \infty} n^{-1} \log Y_{n}=\sum_{1}^{r} p_{j} \log p_{j}\) almost surely. (In information theory, the \(X_{i}\) 's are considered as the output of a source of digital signals, and \(-\sum_{i}^{r} p_{j} \log p_{j}\) is called the entropy of the signal.)

A collection or "population" of \(N\) objects (such as mice, grains of sand, etc.) may be considered as a smaple space in which each object has probability \(N^{-1}\). Let \(X\) be a random variable on this space (a numerical characteristic of the objects such as mass, diameter, etc.) with mean \(\mu\) and variance \(\sigma^{2}\). In statistics one is interested in determining \(\mu\) and \(\sigma^{2}\) by taking a sequence of random samples from the population and measuring \(X\) for each sample, thus obtaining a sequence \(\left\\{X_{j}\right\\}\) of numbers that are values of independent random variables with the same distribution as \(X\). The \(n\)th sample mean is \(M_{n}=n^{-1} \sum_{1}^{n} X_{j}\) and the \(n\)th sample variance is \(S_{n}^{2}=(n-1)^{-1} \sum_{1}^{n}\left(X_{j}-M_{j}\right)^{2}\). Show that \(E\left(M_{n}\right)=\mu, E\left(S_{n}^{2}\right)=\sigma^{2}\), and \(M_{n} \rightarrow \mu\) and \(S_{n}^{2} \rightarrow \sigma^{2}\) almost surely as \(n \rightarrow \infty\). Can you see why one uses \((n-1)^{-1}\) instead of \(n^{-1}\) in the definition of \(S_{n}^{2}\) ?

A fair coin is tossed 10,000 times; let \(X\) be the number of times it comes up heads. Use the central limit theorem and a table of values (printed or electronic) of \(\operatorname{erf}(x)=2 \pi^{-1 / 2} \int_{0}^{x} e^{-t^{2}} d t\) to estimate a. the probability that \(4950 \leq X \leq 5050\); b. the number \(k\) such that \(|X-5000| \leq k\) with probability \(0.98\).

If \(\left\\{X_{j}\right\\}\) is a sequence of independent identically distributed random variables with mean 0 and variance 1 , the distributions of $$ \sum_{1}^{n} X_{j} /\left(\sum_{1}^{n} X_{j}^{2}\right)^{1 / 2} \quad \text { and } \quad \sqrt{n} \sum_{1}^{n} X_{j} / \sum_{1}^{n} X_{j}^{2} $$ both converge vaguely to the standard normal distribution.

The function \(f:\left(\mathbb{R}^{*}\right)^{2} \rightarrow[0,+\infty]\) defined by \(f(t, s)=|t-s|\) for \(t, s \in \mathbb{R}\), \(f(\infty, t)=f(t, \infty)=+\infty\) for \(t \in \mathbb{R}\), and \(f(\infty, \infty)=0\) is lower semicontinuous.