Chapter 2: Problem 9
Let $$ \mathbf{P}=\left\|\begin{array}{cc} 1-a & a \\ b & 1-b \end{array}\right\|, \quad 0
Chapter 2: Problem 9
Let $$ \mathbf{P}=\left\|\begin{array}{cc} 1-a & a \\ b & 1-b \end{array}\right\|, \quad 0
All the tools & learning materials you need for study success - in one app.
Get started for freeIf \(j\) is a transient state prove that for all \(i\) $$ \sum_{n=1}^{\infty} P_{i j}^{n}<\infty $$ uint: Use relation (5.10).
Suppose \(\quad X_{1}, \quad X_{2}, \ldots\) are independent with \(\operatorname{Pr}\left\\{X_{k}=+1\right\\}=p\), \(\operatorname{Pr}\left\\{X_{k}=-1\right\\}=q=1-p\) where \(p \geq q .\) With \(S_{0}=0\), set \(S_{n}=X_{1}+\cdots+X_{n}\) \(M_{n}=\max \left\\{S_{k}: 0 \leq k \leq n\right\\}\) and \(Y_{n}=M_{n}-S_{n} .\) If \(T(a)=\min \left\\{n: S_{n}=a\right\\}\), show Hint: The bivariate process \(\left(M_{n}, Y_{n}\right)\) is a random walk on the positive lattice. What is the probability that this random walk leaves the rectangle
(Continuation). Adding to the notation of Problem 15, let \(\tau\) be the first
time the partial sums \(S_{n}\) deviate \(y\) units from their maximum to date.
That is, let \(\tau=\min \left\\{n: Y_{n}=y\right\\}\). Show that
\(M_{t}=S_{n}+y\) has a geometrie distribution I'r \(\left\langle M_{\mathrm{t}}
\geq a\right\\}=\theta^{n}\) for \(a=0,1, \ldots\) and determine \(0 .\)
llint: \(M_{\mathrm{t}} \geq a\) if and only if \(\max _{0 \leq k \leq T(a)}
Y_{k}
Consider a random walk on the integers such that \(P_{i, l+1}=p, P_{1, i-1}=q\) for all integer i \((0
Every stochastic \(n \times n\) matrix corresponds to a Markov chain for which it is the one-step transition matrix. (By "Stochastie matrix" we mean \(\mathbf{P}=\| P_{y}\) with \(0 \leq P_{i j} \leq 1\) and \(\sum_{j} P_{i j}=1 .\) ) However, not every stochastic \(n \times n\) matrix is the two-step transition matrix of a Markov chain. In particular, show that a \(2 \times 2\) stochastic matrix is the two-step transition matrix of a Markov chain if and only if the sum of its principal diagonal terms is greater than or equal to \(1 .\)
What do you think about this solution?
We value your feedback to improve our textbook solutions.