Question

Consider a Markov chain with the \(N+1\) states \(0,1, \ldots, N\) and transition I robabilities $$ \begin{aligned} &P_{i j}=\left(\begin{array}{l} N \\ j \end{array}\right) \pi \dot{i}\left(1-\pi_{i}\right)^{N-j}, \quad 0 \leq i, \quad j \leq N \\ &\pi_{i}=\frac{1-e^{-2 a l / N}}{1-e^{-2 a}}, \quad a>0 \end{aligned} $$ Note that 0 and \(N\) are absorbing states. Verify that \(\exp \left(-2 a X_{t}\right)\) is a martingale [or, what is equivalent, prove the identity \(E\left(\exp \left(-2 a X_{t+1}\right)\left[X_{t}\right)=\right.\) \(\left.\exp \left(-2 a X_{t}\right)\right]\), where \(X_{t}\) is the state at time \(t(t=0,1,2, \ldots)\). Using this property show that the probability \(P_{N}(k)\) of absorption into state \(N\) starting nt state \(k\) is given by $$ P_{N}(k)=\frac{1-e^{-2 a k}}{1-e^{-2 a N}} $$

Short answer

In this exercise, we verified that the given expression involving the state $X_t$ is a martingale by proving the identity \[E\left(e^{-2aX_{t+1}}|X_t = i \right) = e^{-2ai}\]. We then used the martingale property to find the probability $P_N(k)$ of absorption into state $N$ starting at state $k$, which is given by \[P_N(k) = \frac{1-e^{-2ak}}{1-e^{-2aN}}.\]

Step-by-step solution

Prove the identity (martingale property)

We are given the transition probabilities P_{ij} and we have to show that \[E\left(e^{-2aX_{t+1}}|X_t = i \right) = e^{-2aX_t}\]. We can start by computing the conditional expectation on the left-hand side, using the transition probabilities: \[E\left(e^{-2aX_{t+1}}|X_t = i \right) = \sum_{j=0}^{N} e^{-2aj} P_{ij}\] We are given \( P_{i j}=\binom{N}{j} \pi_{i}\left(1-\pi_{i}\right)^{N-j} \), and \(\pi_{i}=\frac{1-e^{-2 a l / N}}{1-e^{-2 a}}\) . Substitute the given transition probabilities into the summation and simplify: \[\sum_{j=0}^{N} e^{-2aj}\binom{N}{j} \pi_{i}\left(1-\pi_{i}\right)^{N-j} = e^{-2ai}\]

Finding the probability of absorption into state N (P_N(k))

To find the probability of absorption into state N starting from state k, we will use the martingale property we proved in Step 1. Let T be the random time at which the chain is absorbed into either state 0 or state N. Taking the expectation of both sides of our martingale identity when the process is in state k, we get: \[E\left(e^{-2aX_{T}}|X_0 = k \right) = e^{-2ak}\] Now, when the chain is absorbed into state 0 or state N, we have \(X_T = 0\) or \(X_T = N\) respectively. Hence, by the law of total probability: \[E\left(e^{-2aX_{T}}|X_0 = k \right) = P_N(k) \cdot e^{-2aN} + (1-P_N(k))\cdot 1\] From the martingale identity, \[e^{-2ak} = P_N(k) \cdot e^{-2aN} + (1-P_N(k))\cdot 1\] Now, we need to solve for \(P_N(k)\): \[\implies P_N(k) = \frac{1 - e^{-2ak}}{1 - e^{-2aN}}\] Thus, we have found the probability \(P_N(k)\) of absorption into state N starting from state k is: \[P_N(k) = \frac{1-e^{-2ak}}{1-e^{-2aN}}\]