Question

Let \(S_{n}\) denote the time of the \(n\) th event of the Poisson process \([N(t), t \geqslant 0\\}\) having rate \(\lambda\). Show, for an arbitrary function \(g\), that the random variable \(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\) has the same distribution as the compound Poisson random variable \(\sum_{i=1}^{N(t)} g\left(U_{i}\right)\) where \(U_{1}, U_{2}, \ldots\) is a sequence of independent and identically distributed uniform \((0, t)\) random variables that is independent of \(N\), a Poisson random variable with mean \(\lambda t\). Consequently, conclude that $$ E\left[\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right]=\lambda \int_{0}^{t} g(x) d x \quad \operatorname{Var}\left(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right)=\lambda \int_{0}^{t} g^{2}(x) d x $$

Short answer

In summary, we first understood the Poisson processes and the properties of Poisson random variables. Then, we defined the given random variables and showed that they have the same distribution. Finally, we derived the formulas for the expected value and variance of the random variables. The obtained formulas are: $$E\left[\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right]=\lambda \int_{0}^{t} g(x) dx$$ and $$\operatorname{Var}\left(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right)=\lambda \int_{0}^{t} g^2(x) dx$$

Step-by-step solution

Understand the Poisson process and the properties of Poisson random variables

A Poisson process is a stochastic process representing the arrival of events in time at a fixed rate. In this problem, we're given a Poisson process \([N(t), t\geq 0]\) with a rate \(\lambda\). The time of the \(n\)th event in the Poisson process is denoted as \(S_{n}\). A Poisson random variable \(N(t)\) represents the number of arrivals in a fixed interval \((0, t)\) and has a mean and variance of \(\lambda t\).

Define the random variables \(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\) and \(\sum_{i=1}^{N(t)} g\left(U_{i}\right)\)

We are given the random variable \(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\), which is the sum of an arbitrary function \(g\) applied to the time of arrivals in the Poisson process up to time \(t\). We're also given the random variable \(\sum_{i=1}^{N(t)} g\left(U_{i}\right)\), which is the sum of the function \(g\) applied to a sequence of independent and identically distributed uniform \((0,t)\) random variables \(U_1, U_2, \ldots\), which are independent of the Poisson random variable \(N(t)\).

Show that the given random variables have the same distribution

To show that both random variables have the same distribution, we need to compare their distribution functions. The distribution function of a random variable Y, denoted as \(F_Y(y)\), is given by: $$F_Y(y) = P(Y \leq y)$$ Since \(S_{i}\) and \(U_{i}\) are both continuous random variables, the probability distribution functions for both \(g(S_i)\) and \(g(U_i)\) are continuous. Therefore, their corresponding distribution functions, \(F_{g(S_i)}(y)\) and \(F_{g(U_i)}(y)\), can be compared directly. Since the Poisson process is a renewal process, the inter-arrival times of the events are independent and identically distributed. Therefore, we can say that \(g(S_i)\) and \(g(U_i)\) have the same distribution. Additionally, since \(N(t)\) is independent of \(U_1, U_2, \ldots\), the random variables \(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\) and \(\sum_{i=1}^{N(t)} g\left(U_{i}\right)\) also have the same distribution.

Derive the expected value of the random variables

To find the expected value of the random variables, we need to find the expectation of \(g(S_i)\) and \(g(U_i)\). Since \(S_i\) and \(U_i\) have the same distribution: $$E\left[g\left(S_{i}\right)\right] = E\left[g\left(U_{i}\right)\right]$$ Now, we know that \(U_i\) has a uniform distribution on the interval \((0,t)\), so its expectation is: $$E\left[g\left(U_{i}\right)\right] = \int_{0}^{t} g(x) \frac{1}{t} dx$$ Using the property of the expected value of the Poisson process, we get: $$E\left[\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right] = \lambda \int_{0}^{t} g(x) dx$$

Derive the variance of the random variables

To derive the variance of the given random variables, we need to find the variance of \(g(S_i)\) and \(g(U_i)\). Since \(S_i\) and \(U_i\) have the same distribution: $$\operatorname{Var}\left[g\left(S_{i}\right)\right] = \operatorname{Var}\left[g\left(U_{i}\right)\right]$$ Now, we know that \(U_i\) has a uniform distribution on the interval \((0,t)\), so its variance is: $$\operatorname{Var}\left[g\left(U_{i}\right)\right] = \int_{0}^{t} g^2(x)\frac{1}{t} dx - \left(\int_{0}^{t} g(x)\frac{1}{t} dx\right)^2$$ Since the events are independent, the variance of the sum is the sum of the variances: $$\operatorname{Var}\left(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right)=\lambda \int_{0}^{t} g^2(x) dx$$ Hence, we've derived the formulas for the expected value and variance of the given random variables.

Key concepts

Poisson Random Variable

A Poisson random variable is a fundamental concept in probability theory and statistics, particularly useful in modelling the number of events that happen within a fixed interval of time or space. This type of variable associates with processes where events occur randomly and independently of each other, at a constant average rate.

For instance, imagine counting the number of stars that appear within a designated patch of the night sky within an hour, or the number of customers that enter a shop in a day. The characteristic defining such scenarios is their 'randomness', and the Poisson random variable, denoted often as \( N(t) \), encapsulates this stochastic nature. The parameter \( \lambda \) in the Poisson distribution represents the average rate of occurrence for the event, and interestingly, the distribution's mean and variance are both equal to \( \lambda t \), providing a unique property that helps in various analytical computations.

Expected Value

The expected value is a core concept in statistics often referred to as the 'mean' and symbolized by \( E(X) \) for a random variable \( X \). It provides a measure of the central tendency of the distribution, essentially representing the average outcome one would anticipate over a large number of trials.

For a Poisson process, the expected value is a vital tool in quantifying the average number of events. In our example with the sum of functions \( g \) applied to Poisson process events, we arrived at \( E\left[\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right] = \lambda \int_{0}^{t} g(x) dx \), which expresses an average value in terms of the rate \( \lambda \) and the integral of the function applied to the events.

Variance

While expected value gives us a notion of the 'center' of our distribution, variance quantifies the 'spread'. Represented by \( Var(X) \) for a random variable \( X \), it measures the dispersion of the random variable's possible values from its expected value.

Variance is particularly crucial for a Poisson process because it hints at the predictability and stability of the event occurrence over a fixed interval. When we calculated the variance of our function applied within the Poisson process, the result \( Var\left(\sum_{i=1}^{N(t)} g\left(S_{i}\right)\right) = \lambda \int_{0}^{t} g^2(x) dx \) captured not only the average but also the variability of the outcomes. This calculation assists in anticipating fluctuation around the expected number of events in the Poisson process.

Stochastic Process

A stochastic process is as an ensemble of random variables ordered in time, representing systems that evolve randomly over time. It is to probability what a dynamical system is to deterministic processes in physics. This concept is essential in understanding the Poisson process, which is one of the simplest and most widely used types of stochastic processes.

Stochastic processes are powerful tools in modelling scenarios ranging from queueing theory in operational research to the random movement known as Brownian motion in physics. Such processes are defined by their ability to evolve according to probabilistic laws. In our context, the Poisson process is a type of stochastic process which models random events occurring continuously and independently over time.

Compound Poisson Random Variable

Building on the simplicity of the Poisson random variable, the compound Poisson random variable allows us to consider scenarios where the events in a Poisson process lead to outcomes with varying magnitudes. Essentially, it's the sum of random values associated with each event occurring in a given period, with these values being identically distributed and independent of when the events occur.

For instance, if we were observing not just the count of customers entering a store but also recording the amount each customer spends, the total revenue over a day would be represented by a compound Poisson random variable. Calculating expected value and variance for such complex variables involves more advanced integration, as also outlined in the solution to our exercise, which underscores the complexity and usefulness of this concept in real-world applications.