Question

Consider a discrete time Markov chain with states \(0,1, \ldots, N\) whose matrix has elements $$ P_{i j}=\left\\{\begin{array}{cc} \mu_{i}, & j=i-1 \\ \lambda_{i}, & j=i+1 ; \quad i, j=0,1, \ldots, N . \\ 1-\lambda_{i}-\mu_{i}, & j=i \\ 0, & |j-i|>1 \end{array}\right. $$ Suppose that \(\mu_{0}=\lambda_{0}=\mu_{N}=\lambda_{N}=0\), and all other \(\mu_{i}^{\prime} s\) and \(\lambda_{i}\) 's are positive, and that the initial state of the process is \(k\). Determine the absorption probabilities at 0 and \(N\).

Short answer

Given a discrete time Markov chain with a specific transition matrix, the absorption probabilities at states 0 and N can be determined by setting up a system of linear equations for the probabilities of reaching states 0 and N, starting at state i. For each state \(i=1,2,\ldots, k-1, k+1, \ldots, N-1\) and the initial state k, create equations using the transition probabilities \(\mu_i\) and \(\lambda_i\), while keeping in mind the absorbing states have probabilities \(P^0_0 = 1\), \(P^0_N = 0\), \(P^N_0 = 0\), and \(P^N_N = 1\). Solve the linear system using methods such as Gaussian elimination or back-substitution to obtain the absorption probabilities \(P^k_0\) and \(P^k_N\) for reaching states 0 and N from the initial state k.

Step-by-step solution

Understanding the transition matrix

The given matrix has elements defined as follows: $$ P_{ij} = \begin{cases} \mu_i, & \text{if}\ j=i-1\\ \lambda_i, & \text{if}\ j=i+1\\ 1-\lambda_i-\mu_i, & \text{if}\ j=i\\ 0, & \text{if} |j-i|>1 \end{cases} $$ This means that in each time step, the process can either stay at state i with probability \(1-\lambda_i-\mu_i\), move to state \((i-1)\) with probability \(\mu_i\), or move to state \((i+1)\) with probability \(\lambda_i\). From the problem conditions, since \(\mu_0=\lambda_0=\mu_N=\lambda_N=0\), both state 0 and state N are absorbing states, meaning that once the process enters either state 0 or N, it will never leave that state.

Set up the system of linear equations

Now let's define \(P_0, P_N\), the absorption probabilities of reaching states 0 and N respectively. Note that since the system is in state k at time 0, we can define the absorption probabilities for all states \(i=1,2, \ldots, N-1\). Let \(P^i_0\) and \(P^i_N\) denote the absorption probabilities of reaching states 0 and N starting at state \(i\), respectively: - For states \(i=1,2,\ldots, k-1, k+1, \ldots, N-1\): - \(P^i_0 = \mu_i P^{i-1}_0 + (1-\lambda_i-\mu_i)P^i_0 + \lambda_iP^{i+1}_0\) - \(P^i_N = \mu_i P^{i-1}_N + (1-\lambda_i-\mu_i)P^i_N + \lambda_iP^{i+1}_N\) - For the initial state k: - \(P^k_0 = \mu_k P^{k-1}_0 + (1-\lambda_k-\mu_k)P^k_0 + \lambda_kP^{k+1}_0\) - \(P^k_N = \mu_k P^{k-1}_N + (1-\lambda_k-\mu_k)P^k_N + \lambda_kP^{k+1}_N\) - For the absorbing states: - \(P^0_0 = 1\), \(P^0_N = 0\), \(P^N_0 = 0\), \(P^N_N = 1\) Now, we have a system of linear equations where we need to solve for the absorption probabilities, \(P^i_0\) and \(P^i_N\).

Solve the linear system

The system defined in Step 2 can be solved with Gaussian elimination, back-substitution, or any other suitable linear system solving technique. The key is to follow the rules for Markov chains, which allow the process to jump between states \(i\) and \(i+1\) or \(i\) and \(i-1\). The process will then reach either absorbing state 0 or N depending on the initial state k, and the absorption probabilities will depend on the given transition probabilities, \(\mu_i\) and \(\lambda_i\). Our result will be the absorption probabilities \(P^k_0\) and \(P^k_N\), which will give us the probabilities of reaching the absorbing states 0 and N from the initial state k in the discrete-time Markov chain.