Chapter 10

Data \(y_{1}, \ldots, y_{n}\) are assumed to follow a binary logistic model in which \(y_{j}\) takes value 1 with probability \(\pi_{j}=\exp \left(x_{j}^{\mathrm{T}} \beta\right) /\left\\{1+\exp \left(x_{j}^{\mathrm{T}} \beta\right)\right\\}\) and value 0 otherwise, for \(j=1, \ldots, n\). (a) Show that the deviance for a model with fitted probabilities \(\widehat{\pi}_{j}\) can be written as $$ D=-2\left\\{y^{\mathrm{T}} X \widehat{\beta}+\sum_{j=1}^{n} \log \left(1-\hat{\pi}_{j}\right)\right\\} $$ and that the likelihood equation is \(X^{\mathrm{T}} y=X^{\mathrm{T}} \widehat{\pi}\). Hence show that the deviance is a function of the \(\widehat{\pi}_{j}\) alone. (b) If \(\pi_{1}=\cdots=\pi_{n}=\pi\), then show that \(\widehat{\pi}=\bar{y}\), and verify that $$ D=-2 n\\{\bar{y} \log \bar{y}+(1-\bar{y}) \log (1-\bar{y})\\} $$ Comment on the implications for using \(D\) to measure the discrepancy between the data and fitted model. (c) In (b), show that Pearson's statistic (10.21) is identically equal to \(n\). Comment.

Suppose that \(Y\) has a density with generalized linear model form $$ f(y ; \theta, \phi)=\exp \left\\{\frac{y \theta-b(\theta)}{a(\phi)}+c(y ; \phi)\right\\} $$ where \(\theta=\theta(\eta)\) and \(\eta=\beta^{\mathrm{T}} x\). (a) Show that the weight for iterative weighted least squares based on expected information is $$ w=b^{\prime \prime}(\theta)(d \theta / d \eta)^{2} / a(\phi) $$ and deduce that \(w^{-1}=V(\mu) a(\phi)\\{d g(\mu) / d \mu\\}^{2}\), where \(V(\mu)\) is the variance function, and that the adjusted dependent variable is \(\eta+(y-\mu) d g(\mu) / d \mu\). Note that initial values are not required for \(\beta\), since \(w\) and \(z\) can be determined in terms of \(\eta\) and \(\mu\); initial values can be found from \(y\) as \(\mu^{1}=y\) and \(\eta^{1}=g(y)\). (b) Give explicit formulae for the weight and adjusted dependent variable when \(R=m Y\) is binomial with denominator \(m\) and probability \(\pi=e^{\eta} /\left(1+e^{\eta}\right)\).

Suppose that \(y\) is the number of events in a Poisson process of rate \(\lambda\) observed for a period of length \(T\). Show that \(y\) has a generalized linear model density and give \(\theta, b(\theta), \phi\) and \(c(y ; \phi)\)

Show that if \(Y\) is continuous with cumulative hazard function \(H(y)\), then \(H(Y)\) has the unit exponential distribution. Hence establish that \(\mathrm{E}\\{H(Y) \mid Y>c\\}=1+H(c)\), and explain the reasoning behind (10.55).

Let \(Y\) be a positive continuous random variable with survivor and hazard functions \(\mathcal{F}(y)\) and \(h(y)\). Let \(\psi(x)\) and \(\chi(x)\) be arbitrary continuous positive functions of the covariate \(x\), with \(\psi(0)=\chi(0)=1\). In a proportional hazards model, the effect of a non-zero covariate is that the hazard function becomes \(h(y) \psi(x)\), whereas in an accelerated life model, the survivor function becomes \(\mathcal{F}\\{y \chi(x)\\}\). Show that the survivor function for the proportional hazards model is \(\mathcal{F}(y)^{\psi(x)}\), and deduce that this model is also an accelerated life model if and only if $$ \log \psi(x)+G(\tau)=G\\{\tau+\log \chi(x)\\} $$ where \(G(\tau)=\log \left\\{-\log \mathcal{F}\left(e^{\tau}\right)\right\\}\). Show that if this holds for all \(\tau\) and some non-unit \(\chi(x)\), we must have \(G(\tau)=\kappa \tau+\alpha\), for constants \(\kappa\) and \(\alpha\), and find an expression for \(\chi(x)\) in terms of \(\psi(x) .\) Hence or otherwise show that the classes of proportional hazards and accelerated life models coincide if and only if \(Y\) has a Weibull distribution.

By writing \(\sum\left\\{y_{j}-\widehat{g}\left(x_{j}\right)\right\\}^{2}=(y-\widehat{g})^{\mathrm{T}}(y-\widehat{g})\) and recalling that \(y=g+\varepsilon\) and \(\widehat{g}=S y\), where \(S\) is a smoothing matrix, show that $$ \mathrm{E}\left[\sum_{j=1}^{n}\left\\{y_{j}-\widehat{g}\left(x_{j}\right)\right\\}^{2}\right]=\sigma^{2}\left(n-2 v_{1}+v_{2}\right)+g^{\mathrm{T}}(I-S)^{\mathrm{T}}(I-S) g $$ Hence explain the use of \(s^{2}(h)\) as an estimator of \(\sigma^{2}\). Under what circumstances is it unbiased?

For a \(2 \times 2\) contingency table with probabilities $$ \begin{array}{cc} \pi_{00} & \pi_{01} \\ \pi_{10} & \pi_{11} \end{array} $$ the maximal log-linear model may be written as $$ \begin{array}{ll} \eta_{00}=\alpha+\beta+\gamma+(\beta \gamma), & \eta_{01}=\alpha+\beta-\gamma-(\beta \gamma) \\ \eta_{10}=\alpha-\beta+\gamma-(\beta \gamma), & \eta_{11}=\alpha-\beta-\gamma+(\beta \gamma) \end{array} $$ where \(\eta_{j k}=\log \mathrm{E}\left(Y_{j k}\right)=\log \left(m \pi_{j k}\right)\) and \(m=\sum_{j, k} y_{j k} .\) Show that the 'interaction'term \((\beta \gamma)\) may be written \((\beta \gamma)=\frac{1}{4} \log \Delta\), where \(\Delta\) is the odds ratio \(\left(\pi_{00} \pi_{11}\right) /\left(\pi_{01} \pi_{10}\right)\), so that \((\beta \gamma)=0\) is equivalent to \(\Delta=1\)

If \(X\) is a Poisson variable with mean \(\mu=\exp \left(x^{\mathrm{T}} \beta\right)\) and \(Y\) is a binary variable indicating the event \(X>0\), find the link function between \(\mathrm{E}(Y)\) and \(x^{\mathrm{T}} \beta\).

For a generalized linear model with known dispersion parameter \(\phi\) and canonical link function, write the deviance as \(\sum_{j=1}^{n} d_{j}^{2}\), where \(d_{j}^{2}\) is the contribution from the \(j\) th observation. Also let $$ u_{j}(\beta)=\partial \log f\left(y_{j} ; \eta_{j}, \phi\right) / \partial \eta_{j}, \quad w_{j}=-\partial^{2} \log f\left(y_{j} ; \eta_{j}, \phi\right) / \partial \eta_{j}^{2} $$ denote the elements of the score vector and observed information, let \(X\) denote the \(n \times p\) matrix whose \(j\) th row is \(x_{j}^{\mathrm{T}}\), where \(\eta_{j}=\beta^{\mathrm{T}} x_{j}\), and let \(H\) denote the matrix \(W^{1 / 2} X\left(X^{\mathrm{T}} W X\right)^{-1} X^{\mathrm{T}} W^{1 / 2}\), where \(W=\operatorname{diag}\left\\{w_{1}, \ldots, w_{n}\right\\}\) (a) Let \(\widehat{\beta}_{(k)}\) be the solution of the likelihood equation when case \(k\) is deleted, $$ \sum_{j \neq k} x_{j} u_{j}\left(\widehat{\beta}_{(k)}\right)=0 $$ and let \(\widehat{\beta}\) be the maximum likelihood estimate based on all \(n\) observations. Use first-order Taylor series expansion of \((10.65)\) about \(\widehat{\beta}\) to show that $$ \widehat{\beta}_{(k)} \doteq \widehat{\beta}-\left(X^{\mathrm{T}} W X\right)^{-1} x_{k} \frac{u_{k}(\widehat{\beta})}{1-h_{k k}} $$ Express \(\widehat{\beta}_{(k)}\) in terms of the standardized Pearson residual \(r_{P k}=u_{k} /\left\\{w_{k}\left(1-h_{k k}\right)\right\\}^{1 / 2}\) (b) Use a second order Taylor series expansion of the deviance to show that the change in the deviance when the \(k\) th case is deleted is approximately $$ r_{G k}^{2}=\left(1-h_{k k}\right) r_{D k}^{2}+h_{k k} r_{P k}^{2} $$ where \(r_{D k}\) is the standardized deviance residual \(d_{k} /\left(1-h_{k k}\right)^{1 / 2}\). (c) Suppose models \(A\) and \(B\) have deviances \(D_{A}\) and \(D_{B}\). Use (b) to find an expression for the change in the likelihood ratio statistic \(D_{A}-D_{B}\), when the \(k\) th case is deleted. (d) Show that your results (a)-(c) are exact in models with normal errors.

One standard model for over-dispersed binomial data assumes that \(R\) is binomial with denominator \(m\) and probability \(\pi\), where \(\pi\) has the beta density $$ f(\pi ; a, b)=\frac{\Gamma(a+b)}{\Gamma(a) \Gamma(b)} \pi^{a-1}(1-\pi)^{b-1}, \quad 0<\pi<1, a, b>0 $$ (a) Show that this yields the beta-binomial density $$ \operatorname{Pr}(R=r ; a, b)=\frac{\Gamma(m+1) \Gamma(r+a) \Gamma(m-r+b) \Gamma(a+b)}{\Gamma(r+1) \Gamma(m-r+1) \Gamma(a) \Gamma(b) \Gamma(m+a+b)}, \quad r=0, \ldots, m $$ (b) Let \(\mu\) and \(\sigma^{2}\) denote the mean and variance of \(\pi .\) Show that in general, $$ \mathrm{E}(R)=m \mu, \quad \operatorname{var}(R)=m \mu(1-\mu)+m(m-1) \sigma^{2} $$ and that the beta density has \(\mu=a /(a+b)\) and \(s^{2}=a b /\\{(a+b)(a+b+1)\\} .\) Deduce that the beta-binomial density has mean and variance $$ \mathrm{E}(R)=m a /(a+b), \quad \operatorname{var}(R)=m \mu(1-\mu)\\{1+(m-1) \delta\\}, \quad \delta=(a+b+1)^{-1} $$ Hence re-express \(\operatorname{Pr}(R=r ; a, b)\) as a function of \(\mu\) and \(\delta .\) What is the condition for uniform overdispersion?