Question

Each individual in a population of size \(N\) is, in each period, either active or inactive. If an individual is active in a period then, independent of all else, that individual will be active in the next period with probability \(\alpha .\) Similarly, if an individual is inactive in a period then, independent of all else, that individual will be inactive in the next period with probability \(\beta .\) Let \(X_{n}\) denote the number of individuals that are active in period \(n\). (a) Argue that \(X_{n}, n \geqslant 0\) is a Markov chain. (b) Find \(E\left[X_{n} \mid X_{0}=i\right]\). (c) Derive an expression for its transition probabilities. (d) Find the long-run proportion of time that exactly \(j\) people are active. Hint for \((\mathrm{d}):\) Consider first the case where \(N=1\).

Short answer

\(X_n\) is a Markov Chain because it satisfies the memoryless property. The expected number of active individuals in period n given that there are i active individuals initially, is \(E[X_{n} | X_0=i] = i\alpha^n + (N-i)(1-\beta)^n\). The transition probabilities are \(P_{ij} = \begin{cases} (1-\beta)^{j-i} & \text{for}\ j>i \\ \alpha^i \beta^{N-i} & \text{for}\ j=i \\ (1-\alpha)^{i-j} & \text{for}\ j<i \\ \end{cases}\). The long-run proportions of time that exactly \(j\) people are active are found by solving \(\pi_j = \sum_{i=0}^N \pi_i P_{ij}\).

Step-by-step solution

(a) Argue that \(X_{n}\) is a Markov chain

To show that the process \(X_n\) is a Markov chain, we need to prove that the memoryless property holds, meaning that the future state depends only on the current state and not on previous states. Mathematically, we want to show that: \[P(X_{n+1} = j | X_{n} = i, X_{n-1} = i_{n-1}, ... , X_0 = i_0) = P(X_{n+1} = j | X_{n} = i)\] Now, consider the next period where an individual can be either active or inactive. No matter what previous state this individual was in, the same can happen with probability \(\alpha\) if active or with probability \(1 - \beta\) if inactive. Since the status of each individual depends only on their current state and not the states of others or past states, the process satisfies the memoryless property. Therefore, \(X_{n}\) is a Markov chain.

(b) Find \(E[X_{n} | X_0 = i]\)

To find the expected number of active individuals in period n given that there are i active individuals initially, we can use mathematical induction. Base case: For \(n = 0\), the expected number of active individuals is the initial state itself, i.e., \(E[X_0 | X_0 = i] = i\). Induction step: Assuming the result holds for \(n=k\), let's prove it for \(n=k+1\). Using the law of total probability and the Markov property, we have: \[E[X_{k+1} | X_0 = i] = \sum_{j=0}^N j P(X_{k+1} = j | X_0 = i)\] By the Markov property, we have: \[E[X_{k+1} | X_0 = i] = \sum_{j=0}^N j \sum_{l=0}^N P(X_{k+1} = j | X_k = l) P(X_k = l | X_0 = i)\] Now, using the induction hypothesis \(E[X_{k} | X_0=i] = i \alpha^k + (N-i)(1-\beta)^k\), we can substitute the expression for \(E[X_{k+1} | X_0 = i]\): \[E[X_{k+1} | X_0 = i] = i \alpha^{k+1} + (N-i)(1-\beta)^{k+1}\] By induction, the result holds for all \(n \geq 0\).

(c) Derive an expression for the transition probabilities

To derive the transition probabilities, we need to find the probability of moving from state i to state j in one step: \[P_{ij} = P(X_{n+1} = j | X_n = i)\] There can be three cases: 1. If \(j>i\), then at least \(j-i\) inactive individuals have to become active. This occurs with probability \((1-\beta)^{j-i}\). 2. If \(ji \\ \alpha^i \beta^{N-i} & \text{for}\ j=i \\ (1-\alpha)^{i-j} & \text{for}\ j<i \\ \end{cases}\]

(d) Find the long-run proportion of time that exactly \(j\) people are active

For this part, we will use the hint provided and first consider the case when \(N=1\). This is a simple two-state Markov chain with transition probabilities: \[P_{00} = \beta, P_{01} = 1-\beta, P_{10} = 1-\alpha, P_{11} = \alpha\] In the long run, the Markov chain will reach a steady state where the distribution of the number of active individuals no longer changes. This steady state is characterized by the equation: \[\pi_j = \sum_{i=0}^N \pi_i P_{ij}\] For the given case, we have: \[\pi_0 = \pi_0 \beta + \pi_1(1-\alpha)\] \[\pi_1 = \pi_0(1-\beta) + \pi_1\alpha\] Solving this system of equations gives: \[\pi_0 = \frac{1-\alpha}{2-\alpha-\beta}\] \[\pi_1 = \frac{1-\beta}{2-\alpha-\beta}\] Now, for the general case with \(N>1\), the steady-state distribution can be found by solving the equation: \[\pi_j = \sum_{i=0}^N \pi_i P_{ij}\] This will give the steady-state proportions of time that exactly \(j\) people are active in the long run.

Key concepts

Transition Probabilities

Transition probabilities are the heart of any Markov chain analysis. They represent the likelihood of moving from one state to another in a given step. To understand this concept, let's consider individuals in a population that can either be active or inactive. The transition probability, denoted as \(P_{ij}\), is the probability of moving from state \(i\) (current number of active individuals) to state \(j\) (next number of active individuals) in one period.

For our particular scenario, these probabilities depend on the parameters \(\alpha\) and \(\beta\). If the number of active individuals increases, meaning \(j>i\), we must calculate the chances of some inactive individuals becoming active. Conversely, if the number of active individuals decreases (\(j
It's crucial to master these transitions to predict the future states of the system accurately. Transition probabilities provide the foundational information needed to analyze Markov chains and are a powerful tool for modeling a wide range of real-world processes.

Steady State Distribution

The steady state distribution is a fundamental aspect of Markov chains, representing a stable condition where the probabilities of being in each state remain constant over time. In other words, once the system reaches this state, the distribution of future states becomes predictable and doesn't change.

For the case at hand, our goal is to find the long-term behavior when each person can be active or not with certain probabilities. By solving the system of equations that arise from setting the long-term rate of entering a state equal to the rate of leaving that state, we can find the steady state distribution or the long-run proportion of time that there are exactly \(j\) active individuals.

This step is often the most challenging yet rewarding, as it provides insights into the long-term behavior of the system. It's like finding the equilibrium point in a dynamic game, where everything eventually settles.

Probability Models

At its core, a probability model is a mathematical representation of a random process, detailing how outcomes are determined and providing means to calculate the likelihood of different events. Markov chains themselves are sophisticated probability models characterized by the transition probabilities that we explored earlier.

In the context of our exercise, we are looking at a probability model that describes the behavior of individuals in a population switching between being active and inactive. This model allows us to analyze and forecast the state of the population over time, leveraging the Markov property that the future is independent of the past given the present state.

Understanding probability models is instrumental in making sense of various stochastic (random) systems, from population dynamics to financial markets. They serve as predictive tools, helping statisticians, economists, engineers, and scientists to grasp the complexities of randomness and make informed decisions based on the structure and properties of the model.