Question

Let \(\left\\{X_{s}\right\\}\) be a finite-state irreducible Markov chain having the transition probabilities \(\| P_{i j} \mid N_{i, j=1^{*}}\). There then exists a stationary distribution \(\pi\), i.e., a vector \(\pi(1), \ldots, \pi(N)\) satisfying \(\pi(i) \geq 0, i=1, \ldots, N, \sum_{i=1}^{N} \pi(i)=1\), and $$ \pi(j)=\sum_{i=1}^{N} \pi(i) P_{i j}, \quad j=1, \ldots, N $$ Suppose \(\operatorname{Pr}\left\\{X_{0}=i\right\\}=\pi(i), i=1, \ldots, N\). Show that \(\left\\{X_{n}\right\\}\) is weakly mixing, hence ergodic.

Short answer

To prove that the given finite-state irreducible Markov chain is weakly mixing and ergodic, we first assumed the initial probabilities follow the stationary distribution. Then we calculated the joint probability of transitioning from state i to state j after n+1 steps. Taking the limit as n approaches infinity and using the properties of the stationary distribution, we showed that the Markov chain is weakly mixing. Finally, since the Markov chain is finite-state, irreducible and weakly mixing, we concluded that the Markov chain is ergodic.

Step-by-step solution

Definition of weakly mixing and ergodic chains

A Markov chain is weakly mixing if for any two states i and j, the limit of the joint probability of transitioning from state i to state j after n steps converges to the product of their stationary probabilities as n approaches infinity: \[ \lim_{n \to \infty} \operatorname{Pr}\left\\{X_{n}=j \mid X_{0}=i\right\\} = \pi(i)\pi(j), \quad \text{for all } i, j. \] A Markov chain is ergodic if it is irreducible, aperiodic, and weakly mixing.

Show that the given Markov chain is weakly mixing

We are given that the initial probabilities follow the stationary distribution: \(\operatorname{Pr}\left\\{X_{0}=i\right\\}=\pi(i), i=1, \ldots, N\). From the definition of a stationary distribution, we know \[ \pi(j)=\sum_{i=1}^{N} \pi(i) P_{i j}, \quad j=1, \ldots, N. \] Now let us compute the joint probability of transitioning from state i to state j after n+1 steps as follows: \[ \operatorname{Pr}\left\\{X_{n+1}=j \mid X_{0}=i\right\\} = \sum_{k=1}^{N} \operatorname{Pr}\left\\{X_{n+1}=j \mid X_{n}=k\right\\} \operatorname{Pr}\left\\{X_{n}=k \mid X_{0}=i\right\\}. \] Next, notice that \(\operatorname{Pr}\left\\{X_{n+1}=j \mid X_{n}=k\right\\} = P_{kj}\). Therefore, we can rewrite the joint probability as \[ \operatorname{Pr}\left\\{X_{n+1}=j \mid X_{0}=i\right\\} = \sum_{k=1}^{N} P_{kj} \operatorname{Pr}\left\\{X_{n}=k \mid X_{0}=i\right\\}. \] Taking the limit as n approaches infinity, and using the fact that the Markov chain is irreducible, we get \[ \lim_{n \to \infty} \operatorname{Pr}\left\\{X_{n+1}=j \mid X_{0}=i\right\\} = \lim_{n \to \infty} \sum_{k=1}^{N} P_{kj} \operatorname{Pr}\left\\{X_{n}=k \mid X_{0}=i\right\\} = \sum_{k=1}^{N} P_{kj} \pi(k). \] But, from the stationary distribution properties, we have \(\pi(j)=\sum_{k=1}^{N} \pi(k) P_{kj}\). Therefore, \[ \lim_{n \to \infty} \operatorname{Pr}\left\\{X_{n+1}=j \mid X_{0}=i\right\\} = \pi(j) = \pi(i) \pi(j), \quad \text{since } \operatorname{Pr}\left\\{X_{0}=i\right\\} = \pi(i). \] Thus, the Markov chain is weakly mixing.

Conclude that the Markov chain is ergodic

Since the given Markov chain is finite-state and irreducible by assumption, and we showed that it is weakly mixing, we can conclude that the Markov chain is ergodic.

Key concepts

Understanding Stationary Distribution

When studying Markov chains, stationary distribution plays a vital role in understanding the long-term behavior of the system. It is a probability distribution that remains constant as the system evolves over time, meaning it does not change from one step to another.

A vector \( \pi \), which denotes the stationary distribution, can be found where \( \pi(i) \) represents the probability for each state \( i \) such that all values are non-negative and sum up to one (\( \sum_{i=1}^{N} \pi(i)=1 \) ). The key property of the stationary distribution is that if \( \pi \) is the distribution over states at some time, it will be the distribution over states at any future time, assuming the initial state is distributed according to \( \pi \). This is mathematically expressed as:
\[\pi(j)=\sum_{i=1}^{N} \pi(i) P_{i j}, \quad j=1, \ldots, N\]
Thus, understanding stationary distributions is essential for analyzing the equilibrium conditions of Markov processes.

What It Means for a Markov Chain to Be Weakly Mixing

In the context of Markov chains, the term weakly mixing refers to a specific type of behavior where the probability of transitioning from one state to another becomes independent of the initial state given enough time. Essentially, in a weakly mixing Markov chain, the future is not heavily influenced by the distant past.

Formally, a Markov chain is weakly mixing if:
\[\lim_{n \to \infty} \operatorname{Pr}\{X_{n}=j \mid X_{0}=i\} = \pi(i)\pi(j), \quad \text{for all } i, j.\]
When this limit holds, we can say that as the number of steps increases without bound, the joint probability of being in state \( j \) given that we started in state \( i \) converges to the product of their individual stationary probabilities. This concept is closely tied to the convergence to equilibrium and suggests that long-term behavior can be predicted from the stationary distribution alone, regardless of the starting point.

Ergodic Markov Chains and Their Significance

A Markov chain being ergodic has profound implications for the predictability and stability of its long-term behavior. An ergodic Markov chain is one that is both irreducible and weakly mixing, additionally requiring it to be aperiodic—communication between states is not constrained to cycles of a certain length.

For practical purposes, if a Markov chain is ergodic, no matter where you start, the system eventually reaches a behavior that is reflective of the chain's stationary distribution. This results in a robust system where time averages are representative of space averages and long-term predictions are feasible. In summary, an ergodic Markov chain provides a level of predictability and balance that is key for diverse applications such as statistical mechanics, economics, and stochastic modeling.