Question

Let \(\rho(v)=R(v) / R(0)\) be the correlation function of a covariance stationary process \(\left\\{X_{n}\right\\}\), where $$ X_{n+1}=a_{1} X_{n}+a_{2} X_{n-1}+\xi_{n+1} $$ for constants \(a_{13} a_{2}\) and zero mean uncorrelated random variables \(\left\\{\xi_{n}\right\\}\), for which \(E\left[\xi_{n}^{2}\right]=\sigma^{2}\) and \(E\left[\xi_{n} X_{n-k}\right]=0, k=1,2, \ldots .\) Establish that \(\rho(v)\) satisfies the so-called Yule-Walker equations $$ \rho(1)=a_{1}+a_{2} \rho(1), \quad \text { and } \quad \rho(2)=a_{1} \rho(1)+a_{2} $$ Determine \(a_{1}\) and \(a_{2}\) in terms of \(\rho(1)\) and \(\rho(2)\)

Short answer

In summary, for the covariance stationary process \(X_{n+1}=a_{1} X_{n}+a_{2} X_{n-1}+\xi_{n+1}\), the correlation function \(\rho(v)\) satisfies the Yule-Walker equations: $$ \rho(1) = a_1 + a_2 \rho(1) \\ \rho(2) = a_1 \rho(1) + a_2 \\ $$ Furthermore, the constants \(a_1\) and \(a_2\) can be determined in terms of \(\rho(1)\) and \(\rho(2)\) as follows: $$ a_2 = \frac{\rho(1)(\rho(2)-\rho(1))}{1-\rho(1)^2} \\ a_1 = \frac{\rho(2)-\rho(1) a_2}{\rho(1)} $$

Step-by-step solution

Deriving Covariance relations from the given process equation

To start, we can first multiply both sides of the given equation by \(X_{n-m}\) for some integer \(m\), and then take the expectations: $$ E[X_{n+1}X_{n-m}] = a_1 E[X_nX_{n-m}] + a_2 E[X_{n-1}X_{n-m}] + E[\xi_{n+1} X_{n-m}] $$ Now, we will consider 3 cases: \(m=0\), \(m=1\), and \(m=2\).

Calculate correlation for m=0,1,2 using given conditions

Case 1: \(m=0\) $$ E[X_{n+1} X_n] = a_1 E[X_n^2] + a_2 E[X_{n-1}X_n] + E[\xi_{n+1}X_n] $$ We are given that \(E[\xi_{n+1} X_n] = 0\), so we can rewrite as: $$ E[X_{n+1} X_n] = a_1 E[X_n^2] + a_2 E[X_{n-1}X_n] $$ Case 2: \(m=1\) $$ E[X_{n+1} X_{n-1}] = a_1 E[X_nX_{n-1}] + a_2 E[X_{n-1}^2] + E[\xi_{n+1} X_{n-1}] $$ We are given that \(E[\xi_{n+1} X_{n-1}] = 0\), so we can rewrite as: $$ E[X_{n+1} X_{n-1}] = a_1 E[X_nX_{n-1}] + a_2 E[X_{n-1}^2] $$ Case 3: \(m=2\) $$ E[X_{n+1} X_{n-2}] = a_1 E[X_nX_{n-2}] + a_2 E[X_{n-1}X_{n-2}] + E[\xi_{n+1} X_{n-2}] $$ We are given that \(E[\xi_{n+1} X_{n-2}] = 0\), so we can rewrite as: $$ E[X_{n+1} X_{n-2}] = a_1 E[X_nX_{n-2}] + a_2 E[X_{n-1}X_{n-2}] $$

Expressing Expectations as Correlations

We can now rewrite these expressions in terms of the correlation function \(\rho(v)\) as follows: Case 1: $$ R(1) = a_1 R(0) + a_2 R(1) \implies \rho(1) = a_1 + a_2 \rho(1) $$ Case 2: $$ R(2) = a_1 R(1) + a_2 R(0) \implies \rho(2) = a_1 \rho(1) + a_2 $$

Solving for a1 and a2 in terms of rho(1) and rho(2)

We now have a system of equations in terms of \(a_1\) and \(a_2\): $$ \rho(1) = a_1 + a_2 \rho(1) \\ \rho(2) = a_1 \rho(1) + a_2 \\ $$ We can solve this system of equations for \(a_1\) and \(a_2\): $$ a_2 = \frac{\rho(1)(\rho(2)-\rho(1))}{1-\rho(1)^2} \\ a_1 = \frac{\rho(2)-\rho(1) a_2}{\rho(1)} $$ This gives the values of \(a_1\) and \(a_2\) in terms of \(\rho(1)\) and \(\rho(2)\).

Key concepts

Covariance Stationary Process

A covariance stationary process, often simply called a stationary process, is a concept crucial to understanding time series analysis in the field of statistics and signal processing. The stationary process has a consistent mean, variance, and autocovariance that are independent of time. This means that no matter at what point you start observing the process, these properties remain unchanged.

For instance, if we take any two points within the time series data, the covariance between them only depends on the distance or lag between these points and not on their actual position in time. The ability to predict future values based on past observations is a strong attribute that is facilitated by a process being stationary.

It is also important to note that the concept of stationarity is fundamental when dealing with the Yule-Walker equations. These are a set of linear equations that provide us a way to estimate the parameters of an autoregressive model—a model where future values are regressed on the past values. The example provided in the exercise demonstrates a process that is assumed to be stationary, and it is on this premise that we can apply the Yule-Walker equations to find the constants in the model.

Correlation Function

The correlation function, sometimes referred to as the autocorrelation function, is a tool that measures the linear relationship between different points in a time series separated by a certain lag. Mathematically, it is defined as the expected value of the product of the values of a process at two different times.

The function is used to detect any non-randomness in data, to identify an appropriate model for the data, or to estimate parameters within a model. In the exercise, \( \rho(v) \), represents the normalized correlation function for a given lag \( v \). This function is pivotal in understanding the strength and the form of the relationship a random process has with itself over time.

In practice, the correlation function helps us express expectations of product terms, such as \( E[X_{n+1} X_{n-m}] \), in terms of correlations, which then allows us to make use of the Yule-Walker equations effectively.

Stochastic Processes

A stochastic process is a collection of random variables indexed by time or space—think of it as a random process evolving over time. It's the mathematical counterpart for phenomena that unfold unpredictably and are subject to randomness. Examples include stock market fluctuations, signal noise in communications, and even the movements of particles in physics.

Each realization of a stochastic process could be entirely different from another realization because of the inherent randomness, yet some processes exhibit structure in this randomness. For instance, the time series described by the exercise is a type of stochastic process known as an autoregressive process, where the future value incorporates past values along with some noise.

Understanding stochastic processes involves not just predicting future values but also comprehending the underlying structure and probability laws governing the process. When we characterize processes as 'covariance stationary', it eases their study since a lot can be deduced from a few stationary properties, such as through the use of the Yule-Walker equations.